# Classification of Encrypted IoT Traffic Despite Padding and Shaping

Aviv Engelberg  School of Computer Science
Tel Aviv UniversityRamat Aviv, 69978Israel<EMAIL_ADDRESS>and  Avishai
Wool  School of Electrical Engineering
Tel Aviv UniversityRamat Aviv, 69978Israel<EMAIL_ADDRESS>
###### Abstract.

It is well known that when IoT traffic is unencrypted it is possible to
identify the active devices based on their TCP/IP headers. And when traffic is
encrypted, packet-sizes and timings can still be used to do so. To defend
against such fingerprinting, traffic padding and shaping were introduced. In
this paper we demonstrate that the packet-sizes distribution can still be used
to successfully fingerprint the active IoT devices when shaping and padding
are used—as long as the adversary is aware that these mitigations are
deployed, and even if the values of the padding and shaping parameters are
unknown. The main tool we use in our analysis is the full distribution of
packet-sizes—as opposed to commonly used statistics such as mean and variance.
We further show how an external adversary who only sees the padded and shaped
traffic as aggregated and hidden behind a NAT middlebox can accurately
identify the subset of active devices with Recall and Precision of at least
96%. We also show that the adversary can distinguish time windows containing
only bogus cover packets from windows with real device activity, at a
granularity of $1sec$ time windows, with 81% accuracy. Using similar
methodology, but now on the defender’s side, we are also able to detect
anomalous activities in IoT traffic due to the Mirai worm.

##  1\. Introduction

###  1.1. Motivation and Background

There is a rapid growth in the number of IoT devices (Holst, 2021), which
leads to security risks regarding inference of user activities from IoT device
fingerprinting (Bezawada et al., 2018; Msadek et al., 2019; Shahid et al.,
2018). It is already known that when the traffic is unencrypted it is possible
to deduce the active devices in a network with connected smart devices by
using features in the TCP/IP headers (Meidan et al., 2017). Moreover, as IoT
vendors move to encrypted traffic, there is metadata that is still available:
packet-sizes and timings can also be used in order to infer the identity and
activity of IoT devices in a victim network (Acar et al., 2020; Alshehri et
al., 2020; Trimananda et al., 2020; Velan et al., 2015). As a mitigation,
prior research has suggested padding the emitted packets with additional bytes
and shaping the packet transmission timings to obfuscate the traffic and make
it harder for an adversary to identify the devices (Alshehri et al., 2020;
Apthorpe et al., 2019; Dziubinski and Bandai, 2021; Pinheiro et al., 2021).
Despite such defenses, we show in this paper that even with encryption,
padding and shaping applied for the traffic, it is still possible to identify
the devices and activities in an attacked network—as long as the adversary is
aware that these mitigations are deployed, even if the values of the padding
and shaping parameters are unknown. The main tool we use in our analysis is
the full distribution of packet-sizes—as opposed to commonly used statistics
such as mean and variance.

Using similar methodology, but now on the defender’s side, we are also able to
detect anomalous activities in IoT traffic related to possibly compromised
smart devices (Haji and Ameen, 2021; Jerkins, 2017; Sridharan et al., 2018).

###  1.2. Related Work

Prior works (Acar et al., 2020; Alshehri et al., 2020; Trimananda et al.,
2020; Velan et al., 2015) have shown that identifying active IoT devices is
possible even when all the traffic within the network is encrypted. Acar et
al. (Acar et al., 2020) presented a multi-stage attack for inferring user
activities in a smart home. They extracted many binary features related to the
metadata and employed different machine learning algorithms for their
classification process. Trimananda et al. (Trimananda et al., 2020)
demonstrated a way to detect devices’ ON/OFF events using packet-sizes and
directions based signatures. Stöber et al. (Stöber et al., 2013) showed that
it’s possible to identify a smartphone by deriving features like packet-sizes
and timings from traffic associated with popular applications like Facebook
and WhatsApp. Conti et al. (Conti et al., 2015) showed that the actions a user
takes on social network and email service apps can be deduced from encrypted
traffic on Android mobile devices.

Other papers (Bezawada et al., 2018; Shahid et al., 2018) showed that device
type identification can be done by learning different features associated with
behavioral patterns of the smart devices. Some authors (Dong et al., 2020;
Wang et al., 2020) utilized deep learning techniques such as Convolutional
Neural Networks (CNN) and Long Short-Term Memory (LSTM) to identify devices
and recognize specific user commands. Junges et al. (Junges et al., 2019)
developed a signature for every activity and used it to identify the user
activities in the network.

It has been suggested by several papers (Dyer et al., 2012; Pinheiro et al.,
2018; Xiong et al., 2018) that padding and traffic shaping can help hide the
user activities in the network, as they change both the packet-sizes and their
rates within the traffic. Such modifications to the packet-sizes and timings
prevent most of the prior inference attacks that are based directly on these
features. Alshehri et al. (Alshehri et al., 2020) suggested a method of random
padding and showed its efficiency in protecting against packet-size based
inference attacks. Pinheiro et al. (Pinheiro et al., 2021) proposed another
way to pad packets in the network and showed that it can reduce identification
attacks based on packet-size signatures. Shaping the traffic to a
predetermined rate shared by all the devices can also assist in hiding user
activities in the network. Dziubinsky et al. (Dziubinski and Bandai, 2021)
presented how different shaping techniques can significantly enhance the
privacy of the network and decrease the adversary confidence in identifying
user activities. In a sequence of papers Apthrope et al. (Apthorpe et al.,
2017a; Apthorpe et al., 2019, 2017b; Apthorpe et al., 2016) also demonstrated
that traffic rates can be used to infer various device activities and proposed
padding and shaping the traffic as a measure of defense against such attacks.
However, none of these works examined the case where the adversary is aware
that padding and shaping are being applied. As we shall see, even if the
adversary is just aware that some shaping and padding is occurring, and even
without knowing the parameters—the adversary can identify the devices and
activities.

On the anomaly detection side, it is already known that detecting suspicious
behavior within the IoT traffic is possible even when the traffic is
encrypted. Zhang et al. (Zhang et al., 2018) have developed a Deterministic
Finite Automaton (DFA) model for every smart device activity and used it to
identify misbehavior in the traffic, similarly to the Goldenberg-Wool model
(Goldenberg and Wool, 2013) used on industrial control systems’ traffic. Other
papers applied machine learning algorithms in order to identify anomalies in
the traffic (Haji and Ameen, 2021; Jonsdottir et al., 2017; Sridharan et al.,
2018) using statistics such as mean and standard deviation of inter-arrival
times as feature vectors. Sridharan et al. (Sridharan et al., 2018)
implemented modules for both cases of unsupervised and supervised learning and
demonstrated how they can identify different types of attacks. Several works
(Garcia et al., 2021; Lam and Abbas, 2020) have shown how deep learning might
also help in discovering when malicious packets are injected into the traffic
with techniques like Convolutional Neural Networks (CNN) and Auto Encoders
(AE).

###  1.3. Contributions

We show in this paper that the packet-sizes distribution can be successfully
used to identify the active devices from encrypted network traffic even when
padding and shaping are applied, as long as the adversary is aware that these
mitigations are being applied. We are able to do this either from an internal
vantage point, where we can distinguish the device traffic based on Layer 2
data, or from an external vantage point, where we observe traffic that is
hidden behind a middlebox (NAT).

While prior works have focused mostly on deriving metadata signatures
(Alshehri et al., 2020; Trimananda et al., 2020) and statistical metadata
information (Acar et al., 2020) for modeling and identifying devices such as
mean and standard deviation, we rely on the full distributions and use
measures such as the Cosine distance (Lee, 1999; Liao and Xu, 2015) and the
Jensen-Shannon distance (Connor et al., 2013; Lee, 1999) to compare them.

Specifically, we examined the packet-sizes and inter-arrival times
distributions both separately and as a joint distribution, as measures to
identify devices in the network. We show that the packet-sizes and inter-
arrival times distributions are not independent, so using both simultaneously
does not significantly enhance the adversary’s classification power. We found
that the packet-sizes distributions alone is sufficient to identify the active
devices within the network.

To evaluate our tests, we examined traffic from a broad number of state-of-
the-art IoT devices, taken from available open sources (Sivanathan et al.,
2017; Trimananda et al., 2020). We obtained excellent results based on a
learning time of 3 hours and with only 30 minutes of real-time recorded
traffic from the attacked network.

We demonstrate how a local adversary that has only access to the Layer 2 data
exchanged in the LAN can identify the active devices. We further show how an
external adversary who only sees the padded and shaped traffic as aggregated
by the middlebox can perfectly identify the dominating active devices, and can
also infer the subset of active devices with exact identification rate of 75%
and with Recall and Precision of at least 96%.

Next we show that the adversary does not need to know the exact padding and
shaping parameters—fairly rough estimates suffice; and we present simple
methods the adversary can use to estimate the parameters. We also show that
the adversary can distinguish time windows containing only cover (i.e., bogus
shaped) packets from windows with real device activity, at a granularity of
$1sec$ time windows, with 81% accuracy.

Moreover, in the Appendix we present how the packet-sizes distribution can
also be used in the defensive side for achieving novel anomaly detection
methods against an active adversary that injects malicious packets into the
traffic. To check our proposed anomaly detection methods, we inject real
malicious Mirai attacks packets taken from (McDermott et al., 2018) into the
IoT traces. Unlike prior works, that relied upon distribution statistics,
using the full distribution as a feature vector our methods do not suffer from
false alarms, and have a Precision of 100% and a Recall of at least 89% for
all the devices.

##  2\. Preliminaries

###  2.1. Padding And Shaping

Obfuscating LAN network traffic relies on two methods: shaping the traffic,
i.e., changing transmission times of real packets and injecting cover packets
at other times; and padding the emitted packets, meaning the original packet
has additional bytes appended.

It was already discussed by Apthorpe et al. (Apthorpe et al., 2019) that one
can inject cover packets at a constant rate during the whole active time of
the devices. Independent Link Padding (ILP) does exactly that. Specifically,
it makes sure that all the traffic coming out of the devices is emitted in the
same predetermined rate shared by all the devices (injecting cover packets if
needed) and that all the packets are either padded to some constant large
enough packet-size or their new padded size is drawn from some packet-sizes
distribution. When ILP is used over all the devices it becomes impossible to
distinguish between them. Hence, this kind of protection results in high
privacy as no actual inference can take place. The only possible inference an
external adversary might be able to make is the number of active devices.

However, besides providing high privacy against inference attacks, ILP
produces a very high bandwidth overhead. Thus, it was suggested to use
mitigations that balance between good privacy and reasonable overhead. For
that purpose, we consider the Stochastic Padding (STP) method, first suggested
by Apthorpe et al. (Apthorpe et al., 2019) (see the pseudo code in Algorithm
1). In this kind of padding and shaping, the original traffic is shaped and
additional packets are injected in order to create periods of same duration
time $T$ with constant predetermined packet rate $R$. Some periods consist of
real traffic of the device (mixed with additional possible cover packets) and
other periods consist entirely of cover packets. Cover packets are injected
stochastically using a predetermined probability $q$. A unique bit flag is
included inside the encrypted contents or header of the packet, indicating
whether this packet is associated with real traffic or if it is a cover packet
and thus should be ignored by the cloud servers and other devices.

injectStart$\leftarrow$ 0

injectEnd $\leftarrow$ 0

while  _t $\geq$ 0_ do

if  _t mod T == 0 and randomInjection(q)==True_ then

injectOffset $\leftarrow$ random(0,T)

if  _t+injectOffset $>$ injectEnd_ then

injectStart$\leftarrow$ t+injectOffset

injectEnd $\leftarrow$ injectStart+T

else

injectEnd $\leftarrow$ injectEnd+T

end if

end if

if  _injectStart $<$ t $<$ injectEnd_ then

for  _t’ in range(injectStart,injectEnd)_ do

if  _userActivityOccurs(t’)_ then

packetSize $\leftarrow$ realPacketSizeAtTime(t’)

else

packetSize $\leftarrow$ getRandom(distribution)

end if

paddedSize $\leftarrow$ packetSize+random(1,W)

emitSize(roundUp(t’,1/R),paddedSize)

end for

else

if  _userActivityOccurs(t)_ then

injectStart $\leftarrow$ t

injectEnd $\leftarrow$ t+T

for  _t’ in range(injectStart,injectEnd)_ do

if  _userActivityOccurs(t’)_ then

packetSize $\leftarrow$ realPacketSizeAtTime(t’)

else

packetSize $\leftarrow$ getRandom(distribution)

end if

paddedSize $\leftarrow$ packetSize+random(1,W)

emitSize(roundUp(t’,1/R),paddedSize)

end for

end if

end if

end while

Algorithm 1 STP($q$, $T$, $R$, $W$, distribution)

Apthorpe et al. (Apthorpe et al., 2019) did not specify how one should pad the
original packets and how to choose the cover packet-sizes. Several padding
methods were already reviewed in previous works such as (Alshehri et al.,
2020; Pinheiro et al., 2021). In our work we mainly follow the Random Padding
method suggested by Alshehri et al. (Alshehri et al., 2020), which pads every
packet by a random number of bytes between 1 and a parameter $W$. Random
Padding is claimed to provide approximate differential privacy and is useful
against attacks that use packet-sizes features (Alshehri et al., 2020;
Pinheiro et al., 2019; Trimananda et al., 2020). Unless indicated otherwise,
we follow the default parameter value suggested by Alshehri et al. (Alshehri
et al., 2020) of $W=80$ and a shaping probability of $q=0.1$ which was found
to be an effective balance between privacy and bandwidth overhead against the
adversary by Apthorpe et al. (Apthorpe et al., 2019). We also investigate the
so called “Level-100” of padding suggested in (Pinheiro et al., 2021).

Besides choosing the padding and shaping, in our version of STP we elected to
draw the size (before padding) of the cover traffic packets from the device’s
real activities packet-sizes distribution. This makes it more difficult for
the adversary to distinguish between the cover packets and real packets.

To prevent STP from introducing latency we make sure that the shaped traffic
rate $R$ is higher than the rate of any IoT device’s real traffic, and that
the time period $T$ is longer than any device’s duration of real activity. For
our tested devices we used $T=1sec,q=0.1$ and set the rate $R$ to be $100$
packets per second (pps).

###  2.2. Adversary Model

We assume that the adversary knows which devices brands may possibly exist in
the attacked network: i.e., we assume a “closed-world” model. This is a very
realistic assumption as the number of common device manufacturers is not high,
and the adversary may be able to discover the installed devices by physical
observation or social engineering.

We also assume that the adversary is aware of the STP algorithm
implementation. Combining this assumption with the “closed-world” model we get
that the adversary is able to simulate the IoT traffic in its own lab, with or
without shaping and padding (after estimating the shaping and padding
parameters), and use resulting traffic to learn various possible traffic
models.

We consider 3 kinds of adversaries: the local adversary, the external
adversary (Acar et al., 2020; Apthorpe et al., 2019) and the active adversary
(Haji and Ameen, 2021; Jonsdottir et al., 2017; Sridharan et al., 2018).

The local adversary has a passive sniffer on the LAN. Examples for such an
adversary are a compromised IoT device or a WiFi eavesdropper that can observe
the network traffic devices in the LAN exchange between each other. Assuming
that the packets in the attacked network are encrypted by both the WiFi
encryption (WPA) and the network protocols (SSL/TLS), the only reliable
information that the local adversary has access to are packet-sizes, timings
and layer 2 data. Layer 2 data can be used to separate the traffic of IoT
devices from each other - however it can be modified easily so we do not rely
on it for device identification.

In contrast to the local adversary, an external adversary may be a malicious
ISP: the adversary is not within the range of the LAN and cannot see the
internal MAC addresses of devices, but only the MAC address of the middlebox.
We assume that the middlebox is also performing NAT, so the external adversary
sees the aggregated traffic of all the active devices rather than individual
per-device traffic flows. Again, assuming that the traffic is encrypted,
including layer 3&4 headers, the adversary only has access to the packet-sizes
and timing of the aggregated traffic leaving the middlebox gateway.

The goal of both local and external adversaries is to identify the devices
within the network and deduce when they are active, in order to infer user
behaviors. Through this identification, they can infer user activity, such as
“if the motion detector is active - then the user is at home and awake”.

Whereas the local and external adversaries are passive and try to infer device
identity and activity by sniffing traffic, the active adversary injects
malicious packets into the traffic. An example is an IoT device that is taken
over by malware such as Mirai (Jerkins, 2017; Antonakakis et al., 2017; Ngo et
al., 2020). Similarly to the local adversary, the active adversary also sees
the traffic exchanged by the devices in the LAN.

###  2.3. Methodology

We assume that the adversary analyzes both the normal traffic and the shaped
and padded traffic of all the possible IoT devices, under lab conditions. The
adversary then extracts features on which to base the device identification in
a real-time scenario at a given time in the victim network.

Since the local and external adversaries’ goal is to classify the observed
traffic among the possible device brands, similarly to (Perdisci et al.,
2020), in our analysis we use confusion matrices to evaluate the performance.
Each cell ($i$,$j$) in the confusion matrix is associated with the adversary’s
learnt model of device $i$ compared to the actual observed traffic associated
with device $j$. The smaller the value in the cell, the more similar the two
models are. When shown graphically we shade the cells in the matrix so darker
colors indicate larger values. Therefore, the ideal confusion matrix has the
minimal value per column (with the lightest color) on the diagonal—which means
that the device is most likely when comparing to its own learnt model than
when comparing to all the other devices’ learnt models. We define a measure
called diagonal-rate of a confusion matrix which measures the fraction of
columns that have the minimum value on the diagonal cell. We would like to get
a high diagonal-rate, as close to $100\%$ as possible.

Unlike most previous work, we do not rely on statistics like mean and variance
in our model. Instead we use the whole distribution of packet-sizes as a
feature vector $v$. In other words, for each packet-size $s$ the feature
$v[s]$ is the frequency of packets with size $s$.

To compare a learnt model to a tested one we use the Cosine distance (Lee,
1999; Liao and Xu, 2015), which is a measure bounded between 0 and 1 based on
the cosine of the angle $\theta$ between two vectors associated with the
compared distributions’ feature vectors:

$cosine\\_distance(u,v)=1-cos(\theta)=1-\frac{u\cdot v}{\|u\|\|v\|}$

While the adversary can record lab traffic for as long as necessary to train
the models, it is advantageous to limit the duration of time that the
adversary would need to record from the victim network. During preliminary
investigation we checked our analysis with different durations of real-time
test recording, to achieve the desired classification results. We found that
typically 30 minutes of recorded victim traffic suffice (details omitted).
Therefore in all the following sections, unless otherwise specified, we use
test recordings of 30 minutes.

###  2.4. The Data Corpora

For our testing we consider 14 different smart devices as displayed in Table
1. The devices’ traffic traces are from the open sources of Sivanathan et al.
(Sivanathan et al., 2017) and those of Trimananda et al. (Trimananda et al.,
2020). Every device has recorded network traces lasting many hours. We use
separate parts of these traces for learning traffic models as the adversary
and for testing against models of other devices.

In addition, we utilize the same traces to develop and evaluate anomaly
detection models. To model injected malicious traffic, we use the data from
(McDermott et al., 2018) which provides captures from several state-of-the-art
malicious attacks traffic, including the Mirai botnet attacks. Mirai malware,
as presented in (Antonakakis et al., 2017; Jerkins, 2017; Ngo et al., 2020)
detected and then compromised IoT devices over the internet by exploiting the
fact that many users do not change the default device’s vendor credentials. In
our work, we modeled 3 types of activities that Mirai exhibited: (1) the first
stage of infecting a device and connecting to the C&C malicious server, (2) a
SYN flood attack and (3) a DNS attack.

Table 1. The IoT devices we classified and their data sources Device Number |  Device Name |  Origin Dataset
---|---|---
1 |  TP-Link Smart Plug |  Sivanathan et al. (Sivanathan et al., 2017)
2 |  Belkin Wemo Motion Sensor |  Sivanathan et al. (Sivanathan et al., 2017)
3 |  Amazon Echo |  Sivanathan et al. (Sivanathan et al., 2017)
4 |  Netatmo Weather Station |  Sivanathan et al. (Sivanathan et al., 2017)
5 |  Samsung SmartCam |  Sivanathan et al. (Sivanathan et al., 2017)
6 |  HP Printer |  Sivanathan et al. (Sivanathan et al., 2017)
7 |  TP-Link Day Night Cloud Camera |  Sivanathan et al. (Sivanathan et al., 2017)
8 |  Amazon Plug |  Trimananda et al. (Trimananda et al., 2020)
9 |  D-Link Plug |  Trimananda et al. (Trimananda et al., 2020)
10 |  Rachio Sprinkler |  Trimananda et al. (Trimananda et al., 2020)
11 |  Ring Alarm |  Trimananda et al. (Trimananda et al., 2020)
12 |  Roomba Robot |  Trimananda et al. (Trimananda et al., 2020)
13 |  TP-Link Light Bulb |  Trimananda et al. (Trimananda et al., 2020)
14 |  Belkin Wemo Insight Plug |  Trimananda et al. (Trimananda et al., 2020)

##  3\. Features that Survive Encryption

###  3.1. The Selected Features

It is already known one can infer IoT device activities using machine learning
techniques by extracting many meta-data features out of the tested recording,
as shown in several works (Acar et al., 2020; Meidan et al., 2020; Trimananda
et al., 2020). However, we are interested in a model that uses only features
that allow us to identify active devices despite traffic encryption.

Following (Acar et al., 2020), we examined the packet-size and inter-arrival
time features, as these features are accessible even when traffic is
encrypted. However, unlike prior work rather than using distribution
statistics such as mean, variance, kurtosis etc., we use the full
distributions as feature vectors. Packet-sizes range between 54–1594 bytes
across all of the examined IoT devices, so we did not need bins. Instead we
used a feature vector of at most 1541 frequencies. For inter-arrival times we
used bins of width 1 sec, starting from 1 sec up to 108 sec.

To begin with, we checked the classification power of the packet-sizes and
inter-arrival times distributions separately and then also as a joint
distribution. In our analysis we modeled the devices from Table 1 based on a
learnt recording of $3$ hours and tested them on random 30 minute recordings
from a different day to evaluate how well we modeled the traffic. For the two-
features joint distribution we created 2-dimensional bins where each bin is
associated with a unique 1 second bin and a specific packet-size bin.

###  3.2. Feature Independence

To evaluate the features, we first tested whether the packet-sizes and inter-
arrival times are statistically independent. To do so we used Pearson’s Chi-
Squared test of Independence. We observed the contingency table of both
packet-sizes and inter-arrival times frequency distribution. Each cell in the
contingency table is associated with a unique bin of packet-size and inter-
arrival time. In order to use the Chi-Squared test one has to make sure that
at least 80% of the expected values in the contingency table are at least $5$
and that all of them are at least $1$. To reach these conditions we needed to
merge bins. We started by creating wider bins: for the inter-arrival times we
used bins of 5 sec and for the packet-sizes we used bins of 50 bytes each.
Then we iteratively increase the bins size each time with jumps of 5 sec and
50 bytes until they all met the criteria.

In Table 2 we see for example $5$ of the tested devices, and their derived
Chi-squared values for the distributions. The table shows that for all these
devices the Chi-square value is greater than the critical values at
significance level 95%. Thus, we conclude that the packet-sizes and inter-
arrival times distributions are _not_ independent: Each device has both a
typical size distribution and a typical timing distribution, which implies
that using both features has a limited advantage over the use of each
separately.

###  3.3. Basic Classification Power

Next we evaluated the classification power of the features in the external
adversary model in the scenario of one single dominating device when STP is
applied (for $W=80$, $q=0.1$) in the attacked network. We computed the
confusion matrix using each distribution separately (Figures 1, 2) and using
the joint distribution (Figure 3). As discussed in Section 2.3, a cell $(i,j)$
in the confusion matrix represents the cosine distance measure between the
learnt distribution of device $i$ and tested distribution of device $j$.
Figure 1 shows that the packet-sizes distribution is a good feature to
differentiate devices with a diagonal-rate of 14/14. Each smart device has
quite a unique set of packet-sizes associated with it, derived both from the
brand of that device and from the activities it performing. On the other hand,
using only the inter-arrival times distribution as seen in Figure 2 is less
effective as the diagonal-rate is only $6/14$ which means that this method
would incorrectly classify 8 of our devices.

When comparing Figures 1 (packet-sizes distribution) and 3 (joint
distribution) we see that both have a perfect diagonal-rate of $14/14$, with a
slight advantage for the joint distribution: it has darker (larger value)
cells off the diagonal, which means that the devices there are distinguished
more easily. However the difference between the power of the packet-size
method and the joint size-and-time distribution method is small, thus to
simplify our models in the following sections we focus only on the packet-
sizes distributions.

Figure 1. Confusion matrix - Packet-Sizes Model

Confusion matrix that shows the comparison’s results of the devices using
Packet-Sizes Model, getting a diagonal-rate of 14/14

Figure 2. Confusion matrix - Inter-Arrival Times Model

Confusion matrix that shows the comparison’s results of the devices using
Inter-Arrival Times Model, getting a diagonal-rate of 6/14

Figure 3. Confusion matrix - Joint distribution Model

Confusion matrix that shows the comparison’s results of the devices using
Joint distribution Model, getting a diagonal-rate of 14/14

Table 2. Chi-Squared Test of Independence between the packet-sizes and inter-arrival times distributions Device Name |  Inter-arrival time bins |  Packet-size bins |  Percentage of expected values ¿ 5 |  Degrees of freedom |  Critical value at significance level 95% |  Chi-squared Value
---|---|---|---|---|---|---
TP-Link Plug |  100 |  250 |  83 |  6 |  12.59 |  16.21
Belkin Motion |  50 |  1000 |  87 |  1 |  3.84 |  5.4
Amazon Echo |  10 |  1000 |  100 |  2 |  5.99 |  8.37
Netatmo Weather |  100 |  250 |  90 |  4 |  9.49 |  11.1
Samsung Camera |  5 |  500 |  100 |  15 |  25.1 |  26.89

##  4\. The Local Adversary Model

As discussed in Section 2, the local adversary can filter the observed padded
and shaped traffic by the MAC addresses and attempt to identify which MAC
address belongs to which suspected smart device. To evaluate the adversary’s
ability to do so, we classify the devices based on the packet-sizes
distribution. In this section we assume that the adversary knows the shaping
and padding parameters - in Section 6 we explore whether the adversary needs
to know them accurately.

As mentioned in Section 2 we use the parameters $W=80,T=1sec,R=100pps$ ,
$q=0.1$ (recall Algorithm 1). The adversary trains his model based on 3 hours
of recorded traffic that is shaped and padded with STP using those parameters,
and then a shaped and padded sample (from another day) is classified. We
repeated this experiment with 10 samples and 10 training traces. The confusion
matrix in Figure 4 shows that the adversary can successfully determine which
device is associated with which MAC address: the diagonal-rate is $14/14$.
Thus, even when shaping and padding take place in the LAN we are able to
identify the devices based solely on the packet-sizes distribution.

Figure 4. Confusion matrix - Local Adversary

Confusion matrix that shows the comparison’s results of the devices in the
Local Adversary Model, getting a diagonal-rate of 14/14

##  5\. The External Adversary Model

###  5.1. Identifying an Active Dominating Device

As discussed in Section 2 and in (Acar et al., 2020; Apthorpe et al., 2019),
the external adversary has access to the metadata of the aggregated traffic
coming out of the home gateway which acts as a NAT middlebox. We consider the
same shaping and padding parameters as before $W=80,T=1sec,R=100pps$ ,
$q=0.1$. Therefore the external adversary’s goal is to identify which device,
or group of devices, is active, despite their traffic being aggregated. The
simplest case is when only one dominating device is active at a specific time.
In this case, the analysis is identical to that of the local adversary, and
the results of Figure 1 (see Section 3.3) show: similarly to the local
adversary, the external adversary can also tell which dominating device emits
packets at a specific time.

###  5.2. Identifying the Number of Active Devices

When multiple devices are simultaneously active in the victim network, a
relatively basic goal for the external adversary is to determine their number.
We suggest to do this based on the packet rate produced by each combination of
active devices. The adversary can, in advance, try all possible combinations
of learnt active devices and calculate the expected packet rate as a function
of their number. When it has all the average packet rates for each possible
number of active devices it can set thresholds between them to determine the
number of active devices. Then, at attack time, it would record the victim’s
traffic coming out of the middlebox gateway and measure the packet rate. By
comparing it to the traffic thresholds, it can estimate the number of active
devices.

To develop our model, we take 30 minutes recording of every combination of
active devices and compute the average packet rate for each possible number of
active devices (between 1 to 14). We set the thresholds to be the mid-points
between averages. In Figure 5 we can see an example of the first 4 thresholds
in a network of the 14 devices we evaluated.

To evaluate the model, we tested it on a different 30 minutes recording from
the attacked network - each time of a different combination of active devices,
for all the possible combinations - and compared the number estimated by the
model to the actual number of active devices. We repeated this experiment 10
times, each time with a different trace for learning and testing.

Our results are that the estimated number is correct with accuracy of 63%, and
that the classifier always estimates the number of active devices to be the
actual number of active devices plus/minus 1.

Figure 5. Example of the Classifier finding the number of active devices. If
the adversary observes traffic with rate of 61.3 packets per second (pps) it
concludes that 4 devices are active

Example that shows how the classifier estimate the number of active devices
using the thresholds

###  5.3. Detecting Subset of Active Devices

####  5.3.1. Methodology

The external adversary’s goal is to estimate the subset of active devices as
closely as it can to the true subset. Analogous to the standard definitions of
Precision and Recall, in the subset-detection scenario we define “Precision”
to mean the fraction of truly active devices among the estimated set, and we
define “Recall” to mean the fraction of estimated-to-be-active devices among
the truly active devices. Our presented results in Table 3 refer to the
average Precision and Recall over many tests. We also define the “Exact
Identification” to mean the fraction of tests in which the estimated subset
was precisely the active set (i.e., tests in which Precision $=$ Recall
$=100\%$).

####  5.3.2. Full Comparison Check

The most naive approach is to compare the tested subset’s packet-sizes
distribution to all possible subsets’ packet-sizes distributions. The one
whose similarity to the tested subset is the highest would be the estimated
subset of devices. With $n$ candidate devices this approach would require
comparing to $2^{n}$ distributions, which makes it prohibitive for $n$ larger
than a handful of possible devices.

However, in Section 5.2 we saw that the estimated size by the Number Of Active
Devices Classifier is always at most off by 1 than the true subset’s size. We
can use this observation to reduce the time complexity without losing
accuracy: it’s enough to check only the subsets of devices whose size is at
most off by 1 than the estimated size. With $n=14$, using this optimization to
only try 3 possible sizes of subsets we reduced the average time of
identifying the subset by a factor of 10: from 30sec down to 3sec.

Table 3 (right columns) shows the results for different number of candidate
devices in the network. We see that we get Precision and Recall of at least
96% and Exact Identification of at least 75% for any number of devices in the
network.

####  5.3.3. Fast Scores Based Check (FSBC)

Despite the optimization of only testing subsets of the estimated number $\pm
1$, the Full Comparison Check still takes time that is exponential in $n$ at
test time, so it is only effective for small values of $n$. In this section we
suggest a faster, non-exponential test-time algorithm we call the Fast Scores
Based Check (FSBC) - see Algorithm 2. This algorithm also relies on knowing
the number of active devices $\pm 1$.

The basic building block of FSBC, given an aggregate packet-sizes distribution
and the learnt packet-sizes distribution of a specific device $i$, is to give
a score indicating how likely it is that $i$ is active in the aggregate
distribution. Initially all the devices start with a default score,
arbitrarily set to 1 (minimal score is 0).

FSBC first checks what packet-sizes are unique to device $i$, i.e., packet-
sizes that device $i$ emits but other candidate devices do not (this check can
be done in advance during the learning phase). Because it might happen that
unique sizes with low frequencies of some device in the learnt recording would
still occur in a different device’s distribution in the tested aggregated
recording, for each device FSBC only considers the unique size with the top
frequency. If the top-frequency unique size of device $i$ does not occur in
the tested aggregated recording, this is a strong indication that $i$ is not
active, thus its score is decreased.

Next, the adversary identifies for each device $i$ its common packet-sizes:
these are the $f_{1}$ most frequent packet-sizes, for parameter $0\%\leq
f_{1}\leq 100\%$. Then, for each common size with frequency $s$, FSBC checks
whether there are at least $f_{2}\cdot s$ packets with that size in the tested
aggregated recording, for another parameter $0\%\leq f_{2}\leq 100\%$. If
device $i$’s most common packet-sizes occur less frequently in the aggregated
traffic this is another indication that $i$ is not active, and its score is
decreased.

Algorithm 2 calculates the score as above for all $n$ candidate devices. At
the end, it discards the devices with the lowest scores until it remain with
only the top-scoring estimated number of devices by the Number Of Active
Devices Classifier from Section 5.2.

To evaluate the Fast Scores Based classifier we tested it over all possible
subsets of different devices, and repeated this 10 times, each time with a
different trace of learning and testing. To set the values of $f_{1},f_{2}$ we
ranged their values from $0\%$ to $100\%$ with jump of $5\%$ and compared the
results of the 10-times experiments each time with a different value of
$f_{1},f_{2}$. We got the best results for $f_{1}=80\%,f_{2}=90\%$.

Table 3 (left columns) shows that the Precision, Recall and Exact
Identification are all larger when the number of devices is smaller. We get
Precision and Recall of at least 74% for any number of devices.

Comparing the results of FSBC to the Full Comparison Check in Table 3 we see
that with 5 IoT devices, both methods have Precision and Recall of 97–98%, but
FSBC has a low rate of Exact identification for subsets of 10 devices or more.

However, the Full Comparison Check comes at a cost: while the Fast Scores
Based Check has an initialization time of about 5 seconds to learn the common
and unique packet-sizes for network with 14 devices, the Full Comparison Check
requires an initialization time of about 30. Also, the average time of
identifying the subset (after the initialization is completed) is also much
faster for the Fast Scores Based Check: about 10ms while it’s almost 3 seconds
for the Full Comparison Check. This high ratio between the initialization and
identification times of the Full Comparison and the Fast Scores Based checks
remains the same for any number of devices in the network. Thus we conclude
that FSBC gives the adversary a very good estimate of the subset of active
devices at a fraction of the computational cost of the Full Comparison Check.

scoresList $\leftarrow$ [ ]

for  _d in distributions _ do

score $\leftarrow$ 1

uniqueSize $\leftarrow$ extractTopUniqueSize(d,distributions)

commonSizesList $\leftarrow$ extractCommonSizes(d,$f_{1}$)

total $\leftarrow$ 1 + len(commonSizesList)

if  _ uniqueSize not in testedSubset_ then

score $\leftarrow$ score - 1/total

end if

for  _packetSize in commonSizesList _ do

if  _ testedSubset[packetSize]$<f_{2}\cdot$d[packetSize]_ then

score $\leftarrow$ score - 1/total

end if

end for

scoresList.append(score)

end for

estimatedSize$\leftarrow$numberOfActiveDevicesClassifier(testedSubset)

estimatedSubset$\leftarrow$takeTopScoresDevices(scoresList,estimatedSize)

return estimatedSubset

Algorithm 2 FSBC(distributions[ ], $f_{1}$, $f_{2}$, testedSubset) Table 3. Fast Scores Based + Full Comparison Checks Number of Devices | Fast Scores Based Check | Full Comparison Check
---|---|---
|  Recall (%) |  Precision (%) |  Exact Identification (%) |  Recall (%) |  Precision (%) |  Exact Identification (%)
5 |  97 |  97 |  74 |  98 |  98 |  81
10 |  79 |  83 |  18 |  97 |  97 |  77
14 |  74 |  76 |  4 |  97 |  96 |  75

###  5.4. Results with Level-100 Padding

Another padding method, suggested by Pinheiro et al. (Pinheiro et al., 2021),
is Level-100 Padding (see Table 4). The motivation to use this padding is that
it’s considered to have lower bandwidth overhead. In addition, Level-100
padding is considered to offer better privacy as more packets share the same
packet-sizes. We used the same methods as in Sections 5.1– 5.3 in order to
estimate the efficiency of this measure of defense. As seen in Figure 6, in
the case of a single dominating device we still manage to identify it
correctly both in the case of local adversary and external adversary.

We then tested the effect of Level-100 padding on the Number Of Active Devices
Classifier (Section 5.2) and found out that it didn’t change the results: As
expected, Level-100 padding changes the way we pad the packets but not the
number of sent packets, hence it does not reduce the ability to infer how many
devices are currently active at the victim’s network.

In addition, when trying to detect subsets of active devices as before, we
found out that Level-100 padding has an effect mainly on the exact identifying
of the active subset, but nonetheless we still managed to obtain quite high
measures of Precision and Recall (about 76% - see Table 5), meaning that an
adversary can still get high confidence of the active devices in the victim
network. The only measure that is significantly impacted by Level-100 padding
is the exact identification which dropped to about 10%: this makes sense as
with this kind of padding many packets share the same size and thus detecting
the exact subset of active devices becomes considerably more difficult.

Table 4. Level-100 Padding Original Size(s) | Padded Size
---|---
$s$ $\leq$ 100 | 100
100¡ $s$ $\leq$ 200 | 200
200¡ $s$ $\leq$ 300 | 300
300¡ $s$ ¡ 999 | random size from $s$ to 1000
999$\leq$ $s$ $\leq$ 1399 | random size from $s$ to 1400
1400$\leq$ s $\leq$ 1600 | 1600

Figure 6. Confusion matrix - Level-100 Padding

Confusion matrix that shows the comparison’s results of the devices in the
case of Level-100 Padding, getting a diagonal-rate of 14/14

Table 5. Level-100 padding -
Fast Scores Based + Full Comparison Checks Fast Scores Based Check | Full Comparison Check
---|---
Recall (%) |  Precision (%) |  Exact Identification (%) |  Recall (%) |  Precision (%) |  Exact Identification (%)
54 |  55 |  1 |  77 |  76 |  8

(a) Confusion matrix - Learning with W=60, Testing with W=80

Confusion matrix that shows the comparison’s results of the devices in the
case of Learning with W=60, Testing with W=80, getting a diagonal-rate of
14/14

(b) Confusion matrix - Learning with W=40, Testing with W=80

Confusion matrix that shows the comparison’s results of the devices in the
case of Learning with W=40, Testing with W=80, getting a diagonal-rate of
10/14

Figure 7. Detecting dominating device with different learnt and tested W

Detecting dominating device with different learnt and tested W

##  6\. Estimating the shaping and padding parameters

###  6.1. Identifying Padding Parameter $W$

As mentioned in Section 2.1, we model a case which STP pads the traffic
randomly, using the parameter $W$. In order for the adversary to model the
devices and to identify them accurately, ideally it needs to know $W$. We
evaluated what happens if there is a difference between the actual $W$ and the
value $W$ estimated by the adversary. Figure 7(b) repeats the experiment of
Section 5.1 when the adversary estimates $W$ incorrectly. Figure 7(a) shows
that the adversary can still identify correctly the dominating active devices
when under-estimating W by 20 bytes. Figure 7(b) has a diagonal-rate of 10/14
and shows that a larger under-estimation (of 40) yields to incorrect
classifications.

Now, in order to estimate $W$ with reasonable accuracy, the adversary does the
following: whenever there’s indication that only one device is active (e.g., a
dominating device), the adversary compares the recorded trace with the models
of all of the devices, for all the different values of $W$ it modeled. As
discussed earlier, we know that taking the modeled devices with $W$ at most 20
bytes away from the actual value would yield a true classification of that
device’s identity with high likelihood. Therefore it’s sufficient for us to
find $W$ which is at most 20 bytes off from the actual $W$.

To evaluate the adversary’s power, we simulate the padded traffic of every
device for multiple possible values of the parameter $W$ (testing with jumps
of 40 bytes to aim for $\pm$ 20 accuracy). Then we model the learnt devices
for the different values of $W$. Whenever there is an active dominating device
whose identity is unknown to the adversary, it compares it to the different
models for each value of $W$ we checked. Figure 8(c) shows an example of the
derived models’ columns comparison (using cosine distance) to the tested
device, when the true value is $W=80$ and we simulate with $W=50,90,130$ for
one possible device (in this case, $W=90$ has the best estimate of $W$
assuming device 3 is dominant).

The smallest value’s row index in each column indicates which device is
estimated to be active when we simulate with the associated value of $W$, and
the row index of the minimum of all minima indicate which device is the best
candidate out of all columns. The column associated with this minimum has the
closest value of $W$ to the real value used. In Figure 8(c) we choose $W=90$,
because its column has the minimum value 0.039. Since the jump between tested
$W$ values is 40, we conclude that the true value is in the values $90\pm 20$.

(a) Learning with W=50, Testing with W=80

Column’s values that show the comparison’s results of the devices in the case
of Learning with W=50, Testing with W=80

(b) Learning with W=90, Testing with W=80

Column’s values that show the comparison’s results of the devices in the case
of Learning with W=90, Testing with W=80

(c) Learning with W=130, Testing with W=80

Column’s values that show the comparison’s results of the devices in the case
of Learning with W=130, Testing with W=80

Figure 8. Detecting W - Example when device 3 is the dominating device

Example that show the comparison’s results of the devices for each checked
value of W, when device 3 is the dominating device

Figure 9. Confusion matrix -
Learning with q=0.5, Testing with q=0.1

Confusion matrix that shows the comparison’s results of the devices in the
case of Learning with q=0.5 and Testing with q=0.1, getting a diagonal-rate of
14/14

Figure 10. Estimating the value of tested q over all the devices

Plot that show the average estimated value of q as function of the real value
of q, demonstrating that they are always very close

###  6.2. Identifying Shaping Parameter $q$

Next the adversary would like to estimate the shaping probability $q$ used by
the victim. We again assume that a single (dominating) device is active, build
simulated models for multiple values of $q$ and compare them to the actual
traffic with the unknown $q$. Figure 9 demonstrates that even when we model
with value of $q$ that is very different from the real value used, we still
achieve diagonal-rate of 14/14.

Nonetheless, we still wish to get a reasonable estimate of $q$. We observe
that $q$ directly affects the traffic volume: the higher $q$ is, the higher
the pps rate is.

Therefore we check the traffic volume emitted by the dominant device for
different values of $q$, and set thresholds in the mid-points between them
(similar to what we did for the Number Of Active Devices Classifier in Section
5.2). By checking the actual traffic volume recorded in real-time and
comparing it to the determined thresholds we can estimate $q$.

To evaluate this estimation, we checked values of $q$ that range between 0 and
1 with jump of $0.05$. Figure 10 shows the average estimated $q$ (over all the
devices) as a function of the true $q$. The figure shows that in all cases the
estimation is very close to the true value: we precisely estimate the value of
$q$ with accuracy of 91%, and in all cases we estimate the value of $q$ to be
at most $0.1$ off from the actual value.

##  7\. Detecting Periods with Real Traffic

Suppose now that the external adversary, that knows that shaping and padding
are being employed on the middlebox home gateway, wants to distinguish between
real traffic and cover (false) traffic injected by the STP algorithm. As
before, we focus on using the STP with $q=0.1,W=80,T=1sec,R=100pps$.

Recall that STP breaks up time into _periods_ of length T. If the device is
active in period $j$ then STP shapes the traffic so the packets are emitted
uniformly, at a rate of $R$ pps, with equal inter-packet-arrival times (and
padded lengths). If the device is not active in period $j$, STP randomly (with
probability $q$) emits exactly $R/T$ equally-spaced cover packets in period
$j$.

We assume that the adversary knows $T$, but is not synchronized with the
gateway. Thus, the adversary also splits time into time units of length $T$:
for ease of exposition we call the adversary’s time units “time windows” to
distinguish them from the STP’s “periods”. A time window overlaps two periods.
Notice that time windows can be empty when both STP periods it overlaps
contain no real traffic and STP randomly decided not to inject cover traffic
in either.

Each non-empty window might contain real traffic, or it may contain only cover
traffic. The adversary’s goal is to label the non-empty windows, as it splits
them, into two classes: “only cover traffic” and “real traffic”.

Dividing the training-trace into time windows splits each period into two
windows, so we get more non-empty time windows than non-empty periods,
possible twice as many. With our devices and the STP parameters as before we
observed that 72% of the non-empty periods contain real traffic, while 53% of
non-empty windows contain real traffic. Therefore, the partition to windows
instead of periods yields a more balanced training set, where about half of
the windows are labeled as “real traffic” and the other half as “only cover
traffic”.

For the purpose of classifying the non-empty windows between the two classes,
we create a feature vector for each time window. Since STP shapes the traffic,
each time window effectively has $T/R$ “slots” at which a packet is either
emitted or not. Thus for a time window starting at time $t_{start}$ we define
a feature vector of size of $T/R$ where the $i$th feature ($0\leq i\leq
T/R-1$) represents the size of the packet in the $i$th slot (at time
$t_{start}+i*R$), or $0$ if there is no packet in that slot. Note that, unlike
the packet-sizes distributions we used before, this feature is sensitive to
packet-size order, reminiscent of the DFA model (Zhang et al., 2018).

The adversary uses the labeled windows as a classified training set for both
classes of windows. Then the adversary divides the test trace into time
windows in the same way and labels any non-empty tested window using its
training set.

We tried different machine learning algorithms such as Random Forest (RF) and
Support Vector Machine (SVM) but found that the most effective algorithm is
the KNN algorithm. To set the value of $k$, we applied 10-fold cross
validation on the training set, each time with a different value of $k$, for
$k$ between 1 to 150, and chose the best value $k=119$ that results in the
highest accuracy of the model.

To evaluate the model, we tested it on a different trace and repeated this
process for 10 times, each time with a different subset of active devices. The
results in Table 6 demonstrate that we managed to obtain high measures of
Precision and Recall (above 77%) and accuracy of 81%.

Table 6. Classifying The time windows Using KNN with k=119 Recall (%) |  Precision (%) |  Accuracy (%)
---|---|---
78 |  77 |  81

##  8\. Conclusion

In this paper we demonstrated that the packet-sizes distribution can still be
used to successfully fingerprint the active IoT devices when shaping and
padding are used—as long as the adversary is aware that these mitigations are
deployed, and even if the values of the padding and shaping parameters are
unknown. The main tool we used in our analysis is the full distribution of
packet-sizes—as opposed to commonly used statistics such as mean and variance.
We further showed how an external adversary who only sees the padded and
shaped traffic as aggregated and hidden behind a NAT middlebox can accurately
identify the subset of active devices with Recall and Precision of at least
96%. We also showed that the adversary can distinguish time windows containing
only bogus cover packets from windows with real device activity, at a
granularity of $1sec$ time windows, with 81% accuracy. Using similar
methodology, but now on the defender’s side, we were also able to detect
anomalous activities in IoT traffic due to the Mirai worm.

## References

  * (1)
  * Acar et al. (2020) Abbas Acar, Hossein Fereidooni, Tigist Abera, Amit Kumar Sikder, Markus Miettinen, Hidayet Aksu, Mauro Conti, Ahmad-Reza Sadeghi, and Selcuk Uluagac. 2020.  Peek-a-Boo: I See Your Smart Home Activities, Even Encrypted!. In _Proceedings of the 13th ACM Conference on Security and Privacy in Wireless and Mobile Networks_ _(WiSec ’20)_. Association for Computing Machinery, New York, NY, USA, 207–218.  https://doi.org/10.1145/3395351.3399421
  * Alghushairy et al. (2021) Omar Alghushairy, Raed Alsini, Terence Soule, and Xiaogang Ma. 2021.  A Review of Local Outlier Factor Algorithms for Outlier Detection in Big Data Streams.  _Big Data and Cognitive Computing_ 5, 1 (2021).  https://doi.org/10.3390/bdcc5010001
  * Alshehri et al. (2020) Ahmed Alshehri, Jacob Granley, and Chuan Yue. 2020\.  Attacking and Protecting Tunneled Traffic of Smart Home Devices. In _Proceedings of the Tenth ACM Conference on Data and Application Security and Privacy_ _(CODASPY ’20)_. Association for Computing Machinery, New York, NY, USA, 259–270.  https://doi.org/10.1145/3374664.3375723
  * Antonakakis et al. (2017) Manos Antonakakis, Tim April, Michael Bailey, Matt Bernhard, Elie Bursztein, Jaime Cochran, Zakir Durumeric, J. Alex Halderman, Luca Invernizzi, Michalis Kallitsis, Deepak Kumar, Chaz Lever, Zane Ma, Joshua Mason, Damian Menscher, Chad Seaman, Nick Sullivan, Kurt Thomas, and Yi Zhou. 2017.  Understanding the Mirai Botnet. In _26th USENIX Security Symposium (USENIX Security 17)_. USENIX Association, Vancouver, BC, 1093–1110.  https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/antonakakis
  * Apthorpe et al. (2017a) Apthorpe, Reisman, and Feamster. 2017a.  Closing the Blinds: Four Strategies for Protecting Smart Home Privacy from Network Observers. In _IEEE Workshop on Technology and Consumer Protection (ConPro)_. San Francisco, CA, 1–6.  https://www.ieee-security.org/TC/SPW2017/ConPro/papers/apthorpe-conpro17.pdf
  * Apthorpe et al. (2019) Noah Apthorpe, Danny Huang, Dillon Reisman, Arvind Narayanan, and Nick Feamster. 2019.  Keeping the Smart Home Private with Smart(er) IoT Traffic Shaping.  _Proceedings on Privacy Enhancing Technologies_ 2019, 128–148.  https://doi.org/10.2478/popets-2019-0040
  * Apthorpe et al. (2017b) Noah Apthorpe, Dillon Reisman, Srikanth Sundaresan, Arvind Narayanan, and Nick Feamster. 2017b.  _Spying on the Smart Home: Privacy Attacks and Defenses on Encrypted IoT Traffic_.  Technical Report arXiv:1708.05044 [cs.CR].
  * Apthorpe et al. (2016) Noah Apthorpe, Dillon Reissman, and Nick Feamster. 2016\.  A Smart Home is No Castle: Privacy Vulnerabilities of Encrypted IoT Traffic. In _Workshop on Data and Algorithmic Transparency (DAT)_. New York, NY, 1–6.  http://datworkshop.org/papers/dat16-final37.pdf
  * Bezawada et al. (2018) Bruhadeshwar Bezawada, Maalvika Bachani, Jordan Peterson, Hossein Shirazi, Indrakshi Ray, and Indrajit Ray. 2018.  IoTSense: Behavioral Fingerprinting of IoT Devices.  _arXiv preprint arXiv:1804.03852_ (04 2018).
  * Breunig et al. (2000) Markus Breunig, Hans-Peter Kriegel, Raymond Ng, and Joerg Sander. 2000.  LOF: Identifying Density-Based Local Outliers.  _ACM Sigmod Record_ 29, 93–104.  https://doi.org/10.1145/342009.335388
  * Connor et al. (2013) Richard Connor, Franco Alberto Cardillo, Robert Moss, and Fausto Rabitti. 2013.  Evaluation of Jensen-Shannon Distance over Sparse Data. In _Similarity Search and Applications_ , Nieves Brisaboa, Oscar Pedreira, and Pavel Zezula (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 163–168.
  * Conti et al. (2015) Mauro Conti, Luigi Mancini, Riccardo Spolaor, and Nino Verde. 2015.  Analyzing Android Encrypted Network Traffic to Identify User Actions.  _IEEE Transactions on Information Forensics and Security_ 11 (01 2015), 1–1.  https://doi.org/10.1109/TIFS.2015.2478741
  * Dong et al. (2020) Shuaike Dong, Zhou Li, Di Tang, Jiongyi Chen, Menghan Sun, and Kehuan Zhang. 2020\.  Your Smart Home Can’t Keep a Secret: Towards Automated Fingerprinting of IoT Traffic. In _Proceedings of the 15th ACM Asia Conference on Computer and Communications Security_ (Taipei, Taiwan) _(ASIA CCS ’20)_. Association for Computing Machinery, New York, NY, USA, 47–59.  https://doi.org/10.1145/3320269.3384732
  * Dyer et al. (2012) Kevin P. Dyer, Scott E. Coull, Thomas Ristenpart, and Thomas Shrimpton. 2012.  Peek-a-Boo, I Still See You: Why Efficient Traffic Analysis Countermeasures Fail. In _2012 IEEE Symposium on Security and Privacy_. 332–346.  https://doi.org/10.1109/SP.2012.28
  * Dziubinski and Bandai (2021) Kiana Dziubinski and Masaki Bandai. 2021.  Bandwidth Efficient IoT Traffic Shaping Technique for Protecting Smart Home Privacy from Data Breaches in Wireless LAN.  _IEICE Transactions on Communications_ (02 2021).  https://doi.org/10.1587/transcom.2020EBP3182
  * Garcia et al. (2021) Norberto Garcia, Tomas Alcaniz, Aurora González-Vidal, Jorge Bernal Bernabe, Diego Rivera, and Antonio Skarmeta. 2021.  Distributed real-time SlowDoS attacks detection over encrypted traffic using Artificial Intelligence.  _Journal of Network and Computer Applications_ 173 (2021), 102871\.  https://doi.org/10.1016/j.jnca.2020.102871
  * Goldenberg and Wool (2013) N. Goldenberg and A. Wool. 2013.  Accurate Modeling of Modbus/TCP for Intrusion Detection in SCADA Systems.  _International Journal of Critical Infrastructure Protection_ 6 (2013), 63–75.  http://dx.doi.org/10.1016/j.ijcip.2013.05.001
  * Haji and Ameen (2021) Saad Haji and Siddeeq Ameen. 2021.  Attack and Anomaly Detection in IoT Networks using Machine Learning Techniques: A Review.  _Asian Journal of Research in Computer Science_ (06 2021), 30–46.  https://doi.org/10.9734/ajrcos/2021/v9i230218
  * Holst (2021) Arne Holst. 2021\.  _Number of Internet of Things (IoT) connected devices worldwide from 2019 to 2030_.  https://www.statista.com/statistics/1183457/iot-connected-devices-worldwide/
  * Jerkins (2017) James Jerkins. 2017\.  Motivating a market or regulatory solution to IoT insecurity with the Mirai botnet code. In _Proc. IEEE 7th computing and communication workshop and conference (CCWC)_. 1–5.  https://doi.org/10.1109/CCWC.2017.7868464
  * Jonsdottir et al. (2017) Gudrun Jonsdottir, Daniel Wood, and Rohan Doshi. 2017.  IoT network monitor. In _2017 IEEE MIT Undergraduate Research Technology Conference (URTC)_. 1–5.  https://doi.org/10.1109/URTC.2017.8284179
  * Junges et al. (2019) Pierre-Marie Junges, J. François, and O. Festor. 2019\.  Passive Inference of User Actions through IoT Gateway Encrypted Traffic Analysis.  _2019 IFIP/IEEE Symposium on Integrated Network and Service Management (IM)_ (2019), 7–12.
  * Lam and Abbas (2020) Jordan Lam and Robert Abbas. 2020.  Machine Learning based Anomaly Detection for 5G Networks.  _ArXiv_ abs/2003.03474 (2020).
  * learn developers (2020) Scikit learn developers. 2007 - 2020.  _Unsupervised Outlier Detection using Local Outlier Factor (LOF)- Scikit-learn library_.  https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.LocalOutlierFactor.html
  * Lee (1999) Lillian Lee. 1999\.  Measures of Distributional Similarity. In _Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics on Computational Linguistics_ (College Park, Maryland) _(ACL ’99)_. Association for Computational Linguistics, USA, 25–32.  https://doi.org/10.3115/1034678.1034693
  * Liao and Xu (2015) Huchang Liao and Zeshui Xu. 2015.  Approaches to manage hesitant fuzzy linguistic information based on the cosine distance and similarity measures for HFLTSs and their application in qualitative decision making.  _Expert Systems with Applications_ 42, 12 (2015), 5328–5336.  https://doi.org/10.1016/j.eswa.2015.02.017
  * McDermott et al. (2018) Christopher McDermott, William Haynes, and Andrei Petrovksi. 2018\.  Threat Detection and Analysis in the Internet of Things using Deep Packet Inspection.  _International Journal on Cyber Situational Awareness_ 4 (12 2018), 61–83.  https://doi.org/10.22619/IJCSA.2018.100120
  * Meidan et al. (2017) Yair Meidan, Michael Bohadana, Asaf Shabtai, Juan David Guarnizo, Martín Ochoa, Nils Ole Tippenhauer, and Yuval Elovici. 2017\.  ProfilIoT: A Machine Learning Approach for IoT Device Identification Based on Network Traffic Analysis. In _Proceedings of the Symposium on Applied Computing_ _(SAC’17)_. Association for Computing Machinery, New York, NY, USA, 506–509.  https://doi.org/10.1145/3019612.3019878
  * Meidan et al. (2020) Yair Meidan, Vinay Sachidananda, Hongyi Peng, Racheli Sagron, Yuval Elovici, and Asaf Shabtai. 2020.  A novel approach for detecting vulnerable IoT devices connected behind a home NAT.  _Computers & Security_ 97 (2020), 101968.  https://doi.org/10.1016/j.cose.2020.101968
  * Msadek et al. (2019) Nizar Msadek, Ridha Soua, and Thomas Engel. 2019.  IoT Device Fingerprinting: Machine Learning based Encrypted Traffic Analysis. In _2019 IEEE wireless communications and networking conference (WCNC)_. 1–8.  https://doi.org/10.1109/WCNC.2019.8885429
  * Ngo et al. (2020) Quoc-Dung Ngo, Huy-Trung Nguyen, Van-Hoang Le, and Doan-Hieu Nguyen. 2020.  A survey of IoT malware and detection methods based on static features.  _ICT Express_ 6, 4 (2020), 280–286.  https://doi.org/10.1016/j.icte.2020.04.005
  * Perdisci et al. (2020) Roberto Perdisci, Thomas Papastergiou, Omar Alrawi, and Manos Antonakakis. 2020.  IoTFinder: Efficient Large-Scale Identification of IoT Devices via Passive DNS Traffic Analysis. In _2020 IEEE European Symposium on Security and Privacy (EuroS &P)_. 474–489.  https://doi.org/10.1109/EuroSP48549.2020.00037
  * Pinheiro et al. (2018) Antônio J. Pinheiro, Jeandro M. Bezerra, and Divanilson R. Campelo. 2018.  Packet Padding for Improving Privacy in Consumer IoT. In _2018 IEEE Symposium on Computers and Communications (ISCC)_. 00925–00929.  https://doi.org/10.1109/ISCC.2018.8538744
  * Pinheiro et al. (2019) Antônio J. Pinheiro, Jeandro de M. Bezerra, Caio A.P. Burgardt, and Divanilson R. Campelo. 2019\.  Identifying IoT devices and events based on packet length from encrypted traffic.  _Computer Communications_ 144 (2019), 8–17.  https://doi.org/10.1016/j.comcom.2019.05.012
  * Pinheiro et al. (2021) Antônio J. Pinheiro, Paulo Freitas de Araujo-Filho, Jeandro de M. Bezerra, and Divanilson R. Campelo. 2021\.  Adaptive Packet Padding Approach for Smart Home Networks: A Tradeoff Between Privacy and Performance.  _IEEE Internet of Things Journal_ 8, 5 (2021), 3930–3938.  https://doi.org/10.1109/JIOT.2020.3025988
  * Shahid et al. (2018) Mustafizur R. Shahid, Gregory Blanc, Zonghua Zhang, and Hervé Debar. 2018.  IoT Devices Recognition Through Network Traffic Analysis. In _2018 IEEE International Conference on Big Data (Big Data)_. 5187–5192.  https://doi.org/10.1109/BigData.2018.8622243
  * Sivanathan et al. (2017) Arunan Sivanathan, Daniel Sherratt, Hassan Habibi Gharakheili, Adam Radford, Chamith Wijenayake, Arun Vishwanath, and Vijay Sivaraman. 2017\.  Characterizing and classifying IoT traffic in smart cities and campuses. In _Proc. IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)_. 559–564.  https://doi.org/10.1109/INFCOMW.2017.8116438
  * Sridharan et al. (2018) Ragav Sridharan, Rajib Ranjan Maiti, and Nils Ole Tippenhauer. 2018.  WADAC: Privacy-Preserving Anomaly Detection and Attack Classification on Wireless Traffic. In _Proceedings of the 11th ACM Conference on Security & Privacy in Wireless and Mobile Networks_ (Stockholm, Sweden) _(WiSec ’18)_. Association for Computing Machinery, New York, NY, USA, 51–62.  https://doi.org/10.1145/3212480.3212495
  * Stöber et al. (2013) Tim Stöber, Mario Frank, Jens Schmitt, and Ivan Martinovic. 2013.  Who do you sync you are?: smartphone fingerprinting via application behaviour. In _Proc. 6th ACM conference on Security and privacy in wireless and mobile networks_.  https://doi.org/10.1145/2462096.2462099
  * Trimananda et al. (2020) R. Trimananda, Janus Varmarken, A. Markopoulou, and B. Demsky. 2020.  Packet-Level Signatures for Smart Home Devices. In _Network and Distributed Systems Security (NDSS) Symposium_.
  * Velan et al. (2015) Petr Velan, Milan Cermak, Pavel Celeda, and Martin Drašar. 2015\.  A survey of methods for encrypted traffic classification and analysis.  _International Journal of Network Management_ 25 (07 2015).  https://doi.org/10.1002/nem.1901
  * Wang et al. (2020) Chenggang Wang, Sean Kennedy, Haipeng Li, King Hudson, Gowtham Atluri, Xuetao Wei, Wenhai Sun, and Boyang Wang. 2020\.  Fingerprinting Encrypted Voice Traffic on Smart Speakers with Deep Learning _(WiSec ’20)_. Association for Computing Machinery, New York, NY, USA, 254–265.  https://doi.org/10.1145/3395351.3399357
  * Xiong et al. (2018) Sijie Xiong, Anand D. Sarwate, and Narayan B. Mandayam. 2018\.  Defending Against Packet-Size Side-Channel Attacks in Iot Networks. In _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. 2027–2031.  https://doi.org/10.1109/ICASSP.2018.8461330
  * Zhang et al. (2018) Wei Zhang, Yan Meng, Yugeng Liu, Xiaokuan Zhang, Yinqian Zhang, and Haojin Zhu. 2018\.  HoMonit: Monitoring Smart Home Apps from Encrypted Traffic. In _Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security_ (Toronto, Canada) _(CCS ’18)_. Association for Computing Machinery, New York, NY, USA, 1074–1088.  https://doi.org/10.1145/3243734.3243820

##  Appendix A The Active Adversary Model

###  A.1. Overview

We now change our viewpoint to that of the defender trying to detect abnormal
behavior of a device taken over by an active adversary. For that, we again use
the packet-sizes distributions.

Our goal is to detect whenever that happens. We used 3 different traces of 1
hour of traffic to develop and evaluate our methods - (1) normal-behavior-
trace, (2) validation-trace and (3) test-trace. In the learning phase we only
use the normal-behavior-trace and the validation-trace.

The normal-trace contains only normal-behavior traffic associated with the
device. In order to check our anomaly detection methods, we use simulated
Mirai traffic as captured from the real attack (McDermott et al., 2018),
categorised into the three stages in the Mirai attacks discussed in Section
2.4. For a given device trace and Mirai attack stage, we inject the attack
packets into the regular traffic of the validation-trace of the device to
simulate an active adversary exploiting the IoT network. We use both the
normal-trace and validation-trace to set a threshold to differentiate between
windows with normal behavior traffic to windows containing Mirai abnormal
packets.

In the test phase, we inject the Mirai traffic into the test-trace, and
evaluate our models success-rates in detecting the anomalies. We repeat this
experiment 10 times for each device.

We divided our validation-trace and test-trace into time windows and injected
the malicious attack packets into 50% of the windows in both traces. We set
the length of the time window to be 2 minutes long, so we have 30 windows in
each trace.

In this section we shall compare two detection methods: one based on the Local
Outlier Factor (LOF) Algorithm, and the other one directly on the Jensen-
Shannon distance (JSD) (Connor et al., 2013; Lee, 1999) between packet-sizes
distributions: basic experimentation indicated that using the JSD as distance
measure for both methods gives better anomaly-detection results than the
cosine distance. JSD is a metric that is bounded between 0 to 1, based on the
Kullback-Leibler (KL) divergence between two normalized (i.e., with norm equal
to 1) distributions’ feature vectors which are defined on the same packet-
sizes set $X$:

$KL(p||q)=\sum_{x\in X}p(x)\cdot\log{\frac{p(x)}{q(x)}}$

$JSD(u,v)=\sqrt{\frac{1}{2}KL(u||\frac{u+v}{2})+\frac{1}{2}KL(v||\frac{u+v}{2})}$

KL divergence is asymmetric and is not defined if there exists a packet-size
$x\in X$ such that $q(x)=0,p(x)\neq 0$. JSD distance which is a metric (in
particular it’s symmetric) fixes this and is always well-defined.

###  A.2. The Local Outlier Factor (LOF) Method

The first anomaly detection method we used is the Local Outlier Factor (LOF)
method, known for anomaly detection in many cases in the literature
(Alghushairy et al., 2021; Breunig et al., 2000). We divide the normal-trace
too into 30 time windows of 2 minutes long. The feature vector of a time
window in each trace is its packets-sizes distribution.

To set the threshold we use the following process in the learning phase: we
give a score to each validation-trace window, using the LOF algorithm as
demonstrated in Figure 11(a). Validation-trace windows that are further away
from the normal behavior windows of the normal-trace will get higher scores.
We used the python library (learn developers, 2020)
sklearn.neighbors.LocalOutlierFactor  and specifically its attribute negative-
outlier-factor . Taking its opposite number gives the score.

After doing this, we have a score for every validation window. We then plot
the ROC Curve for the validation trace, based on setting each time a different
score as a threshold. Windows with score higher than the threshold are
considered as abnormal, while windows with lower score are considered to be
normal. We then take as our threshold the value that yields the best accuracy
of classification.

In our experimentation with this method of setting the threshold, we got a
perfect ROC Curve with Area Under Curve (AUC) of 1 on the validation trace,
and the chosen threshold also incurs Equal Error Rate (EER) of 0, which means
that our method can differentiate perfectly between the two classes of windows
in the validation trace.

Last, we use the chosen threshold: each test-trace window gets a score in a
similar way and is classified as abnormal if its score is higher than the
chosen threshold, and normal otherwise. We got Precision of 100% over all the
devices, i.e., we didn’t have any false alarms. Table 7 shows that we have a
Recall of at least 93% for each device.

###  A.3. The Jensen-Shannon (JS) Method

In this method we use directly our Jensen-Shannon distance metric to set the
thresholds. This time we considered the whole learning day as a whole packet-
sizes distribution rather than dividing it to time windows like in the LOF
method. Similarly to the LOF method, we give a score to each validation-trace
window. Figure 11(b) shows that each window gets as score the value of the
jensen-shannon distance between it and the whole normal-trace. Validation-
trace windows with higher distance from the whole normal-trace get higher
score. Like before, we plot the ROC curve and choose the best threshold. We
got again a ROC Curve with AUC of 1 and EER of 0, which means that this method
can also distinguish perfectly between the two types of windows.

As before, we use the chosen threshold in order to classify each window from
the test-trace. We got again Precision of 100% over all the devices, so we
didn’t have any false alarms. Table 7 shows that we have a Recall of at least
89% for each device.

###  A.4. Comparing The Methods

Both methods did not have any false alarms caused by the detection method
(i.e., never alerted about anomaly window when it wasn’t the case) and had
Precision of 100% for all the devices. As displayed in Table 7, the LOF method
has slightly better rates of Recall than the JS method, and both have at least
89% Recall over all the devices. Abnormal windows’ distributions are
significantly different than normal windows’ distributions because the attack
packets injected have common sizes of Mirai attacks that repeat themselves in
a high rate (e.g. DNS attack). Thus, our methods managed to differentiate
between the distributions associated with the normal and abnormal windows.

(a) LOF Method

Figure showing how lof learning process work

(b) JS Method

Figure showing how jensen-shannon learning process work

Figure 11. Scoring validation-trace windows

Showing how each method scores its validaion windows

Table 7. Anomaly Detection - Results Device Name |  Recall - JS Method (%) |  Recall - LOF Method (%)
---|---|---
TP-Link Plug |  100 |  100
Belkin Motion |  95 |  100
Amazon Echo |  94 |  96
Netatmo Weather |  89 |  94
Samsung Camera |  100 |  100
HP Printer |  89 |  93
TP-Link Camera |  96 |  100
Amazon Plug |  89 |  95
D-Link |  90 |  100
Rachio Sprinkler |  89 |  94
Ring Alarm |  93 |  94
Roomba |  93 |  93
TP-Link Bulb |  94 |  94
Wemo Insight Plug |  93 |  100