https://www.clamav.net/ antivirus as attack tests. Their results shows that
their overhead is significantly decreased at the cost of 5% detection
accuracy.

Summerville et al. (2015) (Summerville et al., 2015), propose a deep packet
anomaly inspection that can operate on constrained embedded devices. Their
approach can run both on the network stack or at the device level. Firstly a
bit wise feature extraction is performed on each packet. Network payloads are
treated as a sequence of bytes. Feature extraction operates on overlapping
tuples of bytes, called n-grams. A three dimensional feature space is
constructed out of the ngrams. After the three dimensional representation is
constructed out of the training data it is used as means for anomaly
detection. The authors evaluated the proposed approach against traffic
obtained from two Internet-enabled devices: a weather station and an
interactive networked video camera. These were chosen, respectively, to
represent an IoT device that primarily outputs data, a sensor, and one that
primarily receives control commands, an actuator. They trained a detector for
each of their devices on the specific network behaviour and tested various
attacks against including worm propagation and code injections. Their results
show that due to the very basic actions of the devices their very lightweight
approach is able to easily detect all the attacks with high accuracy, at a
very low overhead.

Liu et al. (2011) (Liu et al., 2011), in their work present an immunity based
detection system for IoT. Artificial immune systems are a recent advance in
computational intelligence, their self adaptation and robustness makes them a
very interesting approach for intrusion detection in the IoT. Artificial
Immune Systems are adaptive systems are proposed by Leandro Nunes et al.
(Castro et al., 2002). They are inspired by theoretical immunology and
observed immune functions, principles and models, which are applied to problem
solving, as such they take specific components of the immune systems as
inputs. The main difficulty in the usage of this approach is the matching from
network data to the input of the immunity algorithms. The authors propose the
following matching 1) Antigen $\longrightarrow$ signature of IoT datagram, 2)
Self element $\longrightarrow$ normal datagram, 3) non-self element
$\longrightarrow$ attack data, 4) immune cell $\longrightarrow$ detector, 5)
immature immune cell $\longrightarrow$ immature detector, 6) mature immune
cell $\longrightarrow$ mature detector, 7) memory immune cell
$\longrightarrow$ memory detector, and 8) immune cells recognised harmful
antigens $\longrightarrow$ detector matches datagram with attacks. Each packet
in the system is converted to a bit string. The evaluation is performed
mathematically analysing how the detector evolves to match the malicious
antigens per traditional immune system approach.

Kim (2015), proposes a physical-layer technique for checking the integrity of
information provided by an intermediate node(relay) in cooperative relay
communication systems. Cooperative relays are a way to enhance the coverage of
communications in wireless sensor networks. As the relay depends on
cooperation between nodes, almost like the game of Chinese whispers, if a
single node acts maliciously it will impact the whole system negatively. The
authors propose a detection methodology to evaluate whether a received message
has been tampered with and its integrity maintained. Integrity is the
trustworthiness of information. The proposed approach exploits the physical
layer signal to detect the presence of false data in the received packet at
the destination. Once the packet provided by the relay is determined to be
modified, then it is discarded to avoid incorrect decoding at the destination.
The author defines the proposed system as a mathematical model, they consider
1) a cooperative relay communication system in which a source wishes to send
its message to a destination with the assistance of a relay,2) assume that the
relay is not able to transmit and receive simultaneously, and 3) that time
synchronisation between the source and relay is maintained. Evaluation is
performed mathematically and get results from simulation. The evaluated the
approach on 1) Detection Error Probability, Average Detection Error, Outage
Probability, and finally they use their formula to find the optimum detection
threshold balancing the outage vs accuracy.

Lee et al. (2013) (Lee et al., 2014), analyse the problem of battery drain DoS
attacks on wireless 6LowPAN networks. The authors assume that each node knows
their own energy consumption and a central gateway manages communication
between nodes. The central gateway pols for energy readings every 0.5s, if the
energy consumption increases above 30% of the expected energy consumption for
any given node an alarm is raised, after the second alarm is raised the node
is removed from the routing table. The authors evaluated their work through
simulation using Qualnet 171717Available at : https://www.scalable-
networks.com/qualnet-network-simulation-software-tool. No analysis was done
for accuracy of detection the authors simple showed that under DoS the energy
consumption will go up significantly and infer that therefore their approach
will be 100% accurate. The authors fail to provide convincing evaluation for
their approach not to mention that their proposed scheme in itself with
polling and energy consumption calculation every 0.5 seconds for each device
wold have massive overhead in itself likely speeding up the drain, they also
assume that devices can self calculate energy consumption which for IoT
devices this is not the case and finally their solution of disconnecting the
device if they consume more energy than expected is in itself dossing the
system and doing the attacker a favour.

Song et al. (2010) (Song et al., 2010), make use of Hidden Markov Models (HMM)
as means for anomaly detection in wireless sensor networks. Since HMMs have
been proven to be too resource intensive for sensor devices the authors adapt
the traditional approach to remove transition probabilities to be replaced
with rules of reachability called Weak-HMMs (W-HMM). Their approach works
because when a HMM is functioning normally it will be orbiting within the
states following the rules, however, if the network is under attack it will
deviate from ordinary orbit into an unknown rule, this will be flagged as an
anomaly. Their algorithm is broken into two stages, 1) training,this is where
the ordinary orbit model is constructed, and 2) detection, the surveillance
nodes collect changes in system parameters and a deviation distance is
calculated, if above the allowed threshold it is marked as an attack. Note
that the do not specified every how often the HMM is recalculated. The authors
simulate their approach in MATLAB and evaluate detection accuracy and false
alarms to show the validity of their approach.

###  7.4. Proposed Collaborative Intrusion Detection Systems for IoT

As more and more devices are connected to the internet, IoT systems grow from
tens of devices to potentially thousands. These vast systems may spawn across
various infrastructures, networks, protocols and geo-locations. To properly
monitor these systems various techniques and systems need to be in place and
work together collaboratively to detect attacks. These new types of systems
have increased the need for researchers to consider approaches that can work
together and tackle different intrusions across the layers. This section
provides a review of these approaches and the full summary can be seen in
Table 7.

Table 7. Summary of Collaborative Intrusion Detection Systems Proposed for IoT Paper | Placement | Algorithm | Env. | Eval. | Detection
---|---|---|---|---|---
Gupta et al. (Gupta et al., 2013) | distributed | CI (multi) | wireless | $\CrossedBox$ | protocol1
Amaral et al. (Amaral et al., 2014) | distributed | signature | IPv6 | $\CrossedBox$ | -
Kasinathan et al. (Kasinathan et al., 2013b) | hybrid | rule-based | 6LoWPAN | $\checkmark$ | DoS
Kasinathan et al. (Kasinathan et al., 2013a) | hybrid | rule-based | 6LowPAN | $\checkmark$ | DoS
Amouri et al. (Amouri et al., 2018) | cross-layer | behaviour | IoT | $\circ$ | -
Hassanzadeh et al. (Hassanzadeh and Stoleru, 2011) | distributed | genetic-algo. | IoT | $\circ,\checkmark$ | $\beta^{1}$
Le et al. (Le et al., 2016) | hybrid | specification | RPL | $\circ$ | topology2
Midi et al. (Midi et al., 2017) | centralised | adaptable | IoT | $\checkmark$ | DoS
Shreenivas et al. (Shreenivas et al., 2017) | hybrid | anomaly | RPL | $\circ$ | insider
Bostani et al. (Bostani and Sheikhan, 2017) | distributed | hybrid | WSN | $\circ,\checkmark$ | routing
Arolkar et al. (Arolkar et al., 2011) | distributed | ant-colony | WSN | $\CrossedBox$ | $\beta^{3}$
Coppolino et al. (Coppolino et al., 2013) | hybrid | hybrid | WSN | $\delta$ | $\beta^{4}$
Abhishek et al. (Abhishek et al., 2018) | distributed | anomaly | IoT | $\circ$ | $\beta^{5}$
Pongle et al. (Pongle and Chavan, 2015) | distributed | anomaly | RPL | $\circ$ | wormhole
Zhang et al. (Zhang et al., 2011) | distributed | immunity | smart-grid | $\delta,\circ$ | $\beta^{6}$
Yu et al. (Yu and Tsai, 2008) | mixed | hybrid | WSN | $\CrossedBox$ | routing
Buennemeyer et al. (Buennemeyer et al., 2008) | distributed | hybrid | IoT | $\checkmark,\alpha$ | battery-DoS
Moyers et al. (Moyers et al., 2010) | distributed | hybrid | IoT | $\CrossedBox$ | -
Evaluation |  |  $\CrossedBox$ \- no evaluation performed
---
$\Sigma$ \- mathematical evaluation performed
$\circ$ \- simulation performed
$\delta$ \- trace evaluation performed
$\checkmark$ \- execution evaluation performed
$\alpha$ \- usability evaluated
Detection |  |  $\beta$ \- custom attack list
---
1Flooding, Port Scanning, Web Exploits
2Sinkhole, Rank, Local repair, DIS, Neighbour
3Sinkhole, Misdirection, passive information gathering
4Sinkhole, sleep exhaustion
5Compromised Gateways
6MITM, replay, spoofing, message dropping, DoS, data leakage

In the work of Gupta et al. (2013) (Gupta et al., 2013), the authors propose a
general IDS for use in wireless and pervasive computing triggered by the
increasing growth of the IoT and CPS. The authors present their view that due
to the increase in connectivity and adaptability the best option for intrusion
detection is to use computational intelligence techniques as means of
detection. The authors present a review of attacks and vulnerabilities
specific of these networks, namely: 1) Access Control Attacks, Access control
attacks are an attempt to penetrate into a wireless network by evading weakly
laid WLAN access control measures, such as 802.1X port access controls and AP
filters 2) Confidentiality attacks, an attempt to illegally intercept the
private and confidential information sent over wireless associations, both in
unencrypted as well as encrypted form using 802.11 or other encryption
protocols, 3) Integrity attacks, forged data frames over wireless networks to
either mislead the recipient or to form a ground to facilitate launching
another type of attack, 4) Authentication Attacks, make use of these attacks
to steal legitimate user identities and credentials to access otherwise
confidential and private networks and services, and 5) Availability Attacks, a
class of Denial of Service (DoS) attacks restricting a wireless server from
providing services to authorised clients because of resource exhaustion by
unauthorised clients. They then proposed a three tiered framework to operate
on such wireless systems as an intrusion monitor against these attacks: 1)
Monitoring the network, 2) Computational Intelligence, 3) clustering and
reporting, this approach allows for lightweight and comprehensible detection.
The authors do not provide an evaluation of their approach.

Amaral et al. (2014) (Amaral et al., 2014), propose a collaborative IDS for
the internet of things. Their proposed collaborative architecture distributes
a mixture of NIDS and HIDS across the system. Each NIDS performs local traffic
snooping, each node in the neighbourhood is in itself a host based IDS. At the
network layer a set of rules is configures for detection purposes. As their
proposed environment is IoT they assume a heterogeneous system and therefore
customised rules are present on each of the NIDS. The authors do not specify
how the host based IDSs are configured. Each of the NIDS report to a central
workstation that collates the data and performs assessment. They construct an
IDS application, that is now available on TinyOS repositories working on UDP
and 6LowPAN. Despite the application being tested as running successfully on
TinyOS, no analysis of accuracy or evaluation of performance is done.

Kasinathan et al. (2013-A) (Kasinathan et al., 2013b), propose a DoS detection
architecture for 6LoWPAN. The authors embed their security solution in ebbits
181818Available at : https://vicinity2020.eu/vicinity/content/ebbits, a
business oriented services architecture for the IoT. They create a
collaborative attack detection mechanism composed of a DoS protection manager
at the network level, combined with distributed IDS probes within the physical
layer. Their centralised approach assumes there is a cloud layer above the
physical network running a network manager (ebbits), their DoS protection
resides as a module within the network manager. Within the physical layer the
NIDS probes monitor all traffic witin their subnetwors. The actual IDS
components within the probes is Suricatta 191919Available at :
https://suricata-ids.org/, a widely popular open source IDS. The alerts from
the IDS are sent onward to the DoS detection engine in the could. The authors
claimed that due to their detection engine residing outside the physical
network it adds a layer of resilience and makes it in itself immune to DoS
attacks. To evaluate their approach the authors implement a physical testbed
of devices, a Linux penetration testing machine using metasploit as well as a
Linux host running suricata. Their pen-testing machine made use of UDP
flooding to target the network, and the suricata was setup with a rule to
detect large amounts of traffic and consequently rat limit the traffic.
Results showed that the IDS successfully alerted of attacks even with multiple
target machines, and by implementing their attack on a more powerful machine
was not overwhelmed by the attack (of five other machines).

Kasinathan et al. (2013-B) (Kasinathan et al., 2013a), the authors expand on
their previous DoS detection technique by construstion 6LowPAN decoders to
work directly with suricata. As suricata was never designed to work with
6LoPan these advancements go a long way to make traditional tools function in
new IoT environements. A similar testbed is constructed as per the previous
work to test the new architecture. The authors made use of Scapy
202020Available at : https://scapy.net/ to construct malicious flooding
packets and evaluated it on their system. The authors proposed approach of
making existing mainstream IDS function and adapt to IoT environments is a
valuable first step to the advancement of the IoT IDS field as a whole.

Amouri et al. (2018) (Amouri et al., 2018), propose a collaborative and
distributive approach to intrusion detection in the IoT. Dedicated sniffers
perform detection at the local level whilst a super node performs system wide
detection based on data from the sniffers. The IDS consists of a two-stage
detection process, local and global. The local detection is achieved via a the
sniffers located at opposite ends of a system. The packet counts from MAC and
network layer are the features used by the classifier to generate classified
instances. Each sniffer will calculate the classified instances at set
intervals. The instances are communicated to the super node. The super node
applies an iterative linear regression to generate a time-based profile called
the accumulated measure of fluctuation for malicious and normal nodes. A
profile of a malicious and a normal node is obtained, and an anomaly is
detected using this. The authors evaluate their system using a simulation,
using Cooja and MATLAB, performance evaluation is done on how efficiently the
nodes are labelled by the sniffers and how the supernodes perform the
algorithms.

Hassanzadeh and Stoleru (2011) (Hassanzadeh and Stoleru, 2011), design a multi
objective IDS that optimise on different circumstances and variable factors.
Models are developed for individual objectives, such as node energy,
permissible delay in reporting data, information contained in reports, and
network coverage. Their approach uses a genetic algorithm , employing a
penalised function, to solve the multi-objective optimisation problem. The
system consists of a resource constrained wireless network with a single base
station. The network is loosely time synchronised and all nodes know their
locations. The base station periodically collects network information and uses
this information to decide which IDS functions to run, and where to run them.
This decision is done periodically, or when network conditions change. The GA
is used to optimise a specific parameter and its output can be used as a
predictive function. The authors evaluated their algorithm through MATLAB
simulations, analysing time for convergence of the algorithm, number of
parameters, and efficiency of generation for different nodes. They authors
evaluated the proposed IDS on a physical testbed, this consisted of a system
of 6 laptops, with constrained resources, the central IDS was running SNORT
and they evaluated performance and detection accuracy. Results show that their
optimisation approach reduced overhead whilst their information exchange was
sufficient for the central node to detect attacks with high levels of
accuracy.

Le et al. (2016) (Le et al., 2016), propose a new specification based IDS for
RPL, with a new methodology to create behaviour specifications. The authors
make use of Cooja network simulator to create traces of network behaviour,
specifically focusing on route establishment and maintenance. The authors then
devise two algorithm to extract a state transition system from these network
traffic datasets. These high level transitions are then used to represent a
specification of positive behaviours in the system. The authors then place the
specification based detectors across the system, with both HIDS on every node
as well as NIDS across the network for full coverage. A clustering
architecture is form with both the NIDS and HIDS cooperating to communicate
with a central sink. To avoid high overhead the system is not set on
promiscuous mode but rather reporting is done periodically. As nodes report
information of its neighbours, a cross-check of the collected data is
required. The authors evaluated their approach using cooja, with an experiment
consisting of 100 nodes in a 600x600 m area, with transmission ranges of 50m.
Evaluation was performed on detection accuracy including and precision, as
well as energy usage. Their results show a small overhead of 6.3% increase in
energy consumption per node, with high accuracy of detection including the
ability to find new attacks thanks to their adaptive specification algorithm.

Midi et al. (2017) (Midi et al., 2017), bring forth Kalis, a system for
knowledge-driven adaptable intrusion detection for the IoT. The authors
proposed approach envisions for their IDS to adapt based on the knowledge
gathered on the system. The Architecture for Kali is the following: 1)
Communication System, the Communication System interfaces with the external
world. Specialised sub-components take care of interacting with traffic on
different protocols. 2) Data Store: the Data Store listens for events from the
Communication System on newly captured packets, manages the history of recent
traffic for modules to access, and logs all traffic on disk or memory, if
required by the user 3) Knowledge Base and Collective Knowledge
Management:This component stores all the available information about the
features of the monitored entities and networks in a unique, centralised
place, and makes this information available to all the parties requiring it,
such as Detection Modules and the Module Manager. The core component of Kalis
is the way they model the knowledge about the system. The knowledge in the
system may take various different forms and is consequently labelled with
different types, e.g. number of components in a cluster is an int whilst
whether the cluster is multi-hoop is a boolean. Each knowledge component has a
creator ID associated with the creating node and an entity field for whom it
refers to. These knowledge components may be multi-level and collaboratively
enhanced. The collective knowledge of the system is the combination of all te
elements created by the various components of the system. This knowledge is
used by the IDS to make informed decisions about relevant information.
Knowledge elements can be static or adaptable (e.g. signal strength). The
final component of the architecture is 4) modules, in Kalis any network
feature-specific or attack-specific functionality is implemented as an
independent module. The Module Manager component coordinates all the modules,
activating/deactivating them as needed, depending on changes in the Knowledge
Base, Modules are divided into sensing modules, which are the core of the
knowledge discovery mechanism and detection modules which analyse the captured
traffic together with the knowledge elements to look for, they implement this
on a TelosB board using TinyOS. For their experiment they create a WSN of 6 of
these nodes, a smart thermostat a smart lock, a smart light bulb an arlo
security system 212121Available at :
https://www.arlo.com/uk/products/arlo/default.aspx and an amazon dash button.
The Kalis node is placed near the centre of the WSN to overhear all traffic.
They perform the following evaluations 1) breath of coverage, 2) performance
of their detection, 3) adaptability of their system to changes, and, 4)
benefits of the collaborative mechanism. To test the system a ICMP dos attack
flood is used. Their evaluation shows good results even when an initial
environment is not specified showing the adaptability, good detection accuracy
compared to SNORT IDS and low overhead for their knowledge based algorithm.

Shreenivas et al. (2017) (Shreenivas et al., 2017), propose new extensions to
their previous IDS SVELTE to detect insider attacks, specifically routing
attacks, in unsupervised systems. To achieve this aim the authors make used of
ETX (Expected Transmissions) metric, ETX is a link reliability metric and
monitoring the ETX value can prevent an intruder from actively engaging
6LoWPAN nodes in malicious activities. A type of attack that is possible in
these kinds of networks is for a malicious insider to advertise a fake ETX
link to trick other nodes in believing it has a strong connection, therefore
modifying routing. The authors therefore propose an IDS based on expected ETX
values, their approach estimates the value for each node and values which are
strongly different get counted as anomalies. As their algorithm relies heavily
on inter node communication it does not cater to multiple insiders. To adapt
to this they propose an extra caveat of geographical hints, to get
geographical insights clusters are made and a knowledge base of neighbourhoods
is created so if several nodes are misbehaving from the previous knowledge
base it can be spotted. Similarly to the precious approach the authors
implement their IDS in Cooja for evaluation. They preform analysis on power
consumption, true positive rate and memory usage. Results show that for this
specific attack the detection has improved, with a slightly higher overhead.

Bostani and Sheikhan (2017) (Bostani and Sheikhan, 2017), propose a hybrid
IDS, both using anomaly and specification based detection, for the IoT. Their
unsupervised clustering approach is used for the anomaly detection, whilst
specifications are used for known attack patterns. To deal with large amount
of data they make use of Googles MapReduce, this approach reduces computation
into smaller tasks on different machines and is ran simultaneously. Their IDS
has the following stages: Stage 1) identifying malicious nodes using
specification, the goal of this stage is identifying the suspicious nodes that
may cause sinkhole and selective-forwarding attacks. To detect Sinkhole
attacks, the authors propose a specification indicating that when a sinkhole
is happening the rate of parent change for a node is increased, if this
happens then a Sinkhole is occurring. To detect selective forwarding attacks,
the authors use the following signature: when a malicious node wants to launch
a selective-forwarding attack, it selectively forwards packets to the root;
so, a preferred parent node can identify the suspicious node by knowing the
approximate number of packets received from each node. Stage 2) anomaly
detection, the anomaly based IDS creates a sample for each source node by
extracting four traffic-related features from the raw received packet of the
source node at each time-slot (a) packet receiving rate; (b) packet dropping
rate; (c) average latency; and (d) maximum hop-count. The MapReduce approach
is used to optimise this process, a sample is sent to each source node to
compute the features in parallel. Stage 3) anomaly detection decision based on
a voting mechanism, in this stage, the proposed framework employs the first
stage results to make a decision about abnormalities detected in the second
stage. The authors create their own WSN simulator for RPL using .NET and c#.
They also implemented their IDS using MATLAB on a standard PC receiving the
trace data from the simulator. Several simulations were run using different
network parameters and detection accuracy was evaluated. As an experiment
several other RPL attacks were evaluated to examine the flexibility of their
IDS. As a final consideration the authors further evaluate their work on a
real-world scenario. They implemented their a system using Waspmote Mote
Runner 222222Available at : http://www.libelium.com/products/waspmote-mote-
runner-6lowpan/ for gateways and nodes. Their setup mimicked a smart city
including, smart lighting, acoustic noise-maps, air quality monitoring ans
waste management. This is available as a smart kit from WaspMote. The IDSs are
implemented in the gateways and evaluation is performed. No results are
discussed on the real-word implementation but rather a feasibility and
implementation discussion is provided.

Arolkar et al. (2011) (Arolkar et al., 2011), focus on proposing a new
biologically inspired algorithm for Intrusion detection is WSN. Their proposed
ant colony based architecture has the following for stages 1) Formation of
cluster without malicious node, To form clusters, the WSN is divided into
different geographical regions G. Within each geographical region G a node N
is chosen randomly and a parameter L that indicates the level of neighbour in
the cluster is decided. 2)Identification of Cluster Head within Clusters,
three random nodes are chosen within the cluster. Whichever node has with
highest resources is then selected as a cluster head. 3) Deployment of Ant
Pheromones on Cluster Heads, cluster heads deploy initial pheromone values
using ant agents on the neighbour cluster heads. The pheromone values are
refreshed based on preset time intervals. 4) Routing and Detecting Malicious
Activities based on Pheromone Values, once the pheromone is deployed over all
the cluster heads, the routing process can begin. The routing process is now a
two stage process. In the first stage, data is sent to respective cluster head
by the source node. In the second stage, the source cluster head, based on the
pheromone value, then analyses possible threats like sinkhole attack,
misdirection and passive information gathering. A sinkhole attack is detected
if any cluster head responds with a very high pheromone value before the
completion of the pheromone refresh time interval. Misdirection is detected if
the received pheromone value is very low even after it is recently refreshed
Passive information gathering is said to be done if energy and pheromone
values do not decrease proportionally. No evaluation is performed.

Coppolino et al. (2013) (Coppolino et al., 2013), propose an adaptive IDS for
attacks on constrained networks. The architecture of the proposed IDS
comprises a Central Agent(CA), which is connected to a number of Local Agents
(LAs)running on the WSN nodes. The Central Agent performs misuse-based
detection activities, while lighter, anomaly-based detection techniques are
used by the Local Agents to spot local symptoms of an intrusion. On the
contrary of other similar approaches that use a fixed anomaly threshold, the
authors adopted a more flexible approach to threshold setting that consists in
estimating the threshold value based on the specific characteristics of the
WSN being monitored. This allows to decrease communication overhead, which are
highly energy consuming for WSN motes. The Central Agent is located outside
the network on a server Local Agents perform local agents perform a
preliminary analysis within the network. When one or more Local Agents detect
a potential attack, an alert is raised and propagated to the Central Agent.
Local Agents have three components 1) Packet Monitor, 2) Data Collector and
3)Detection Engine. The Local Agents have threshold based anomaly as
lightweight detectors. The Central Agent uses a Decision Tree based algorithm.
The IDS has two stages 1) profiling, this is where the Central Agent maps out
the network topology and then normal behaviours are mapped based on reports
from the local agents. The normal behaviour is set by a set of parameters each
with ranges of values. The central agent stores each of these parameters and
its accepted ranges. Stage 2) each Local Agent performs anomaly-based
intrusion detection based on the normal profile built in the previous stage. A
local alert is raised whenever a parameter of the monitored node exceeds the
normal profile threshold; the Central Agent performs misuse detection for the
whole WSN and validates local alerts through the multidimensional
representation of parameters it create din the previous step. The authors
specifically focus on sinkhole attack as well as sleep deprivation attack, an
attack whose focus is to exhaust the resources of a node by not allowing it to
enter sleep mode. Evaluation was performed with different decision tree
approaches on static data purely on false positive and false negative rates.

Abishek et al. (2018) (Abhishek et al., 2018), propose an IDS specifically for
detecting compromised gateways something which is quite difficult in the
context of systems using clusters. The proposed system uses the packet drop
probability as a means to monitor the gateways. As part of this technique all
the IoT devices will track the number of packets dropped at their respective
ends. This is then reported periodically to the end point. Based on deviations
from expected drops the authors are able to detect anomalies using the Maximum
Likelihood estimation. The approach is evaluated using a simulation in MATLAB,
using RMSE they estimate the effectiveness of the algorithm.

Pongle and Chavan (2015) (Pongle and Chavan, 2015), propose an IDS
specifically for the detection of wormhole attacks. Their architecture
envisions two types of modules, centralised and distributed. They both perform
the following operations: i) Neighbor Validation. In this module we are
collecting the neighbour information from all sensor nodes storing them. The
stored neighbours information are verified based on the distance between the
node and that neighbour. If the distance found to be more than transmission
range of node then this module send the victim packet to the information
sender node and to the neighbour whose distance is more than transmission
range. ii) Distance RSSI. This module calculates the distance be-tween two
geographical coordinates. It also convert the RSSI (Received Signal Strength
Indicator) value to distance and vice versa. iii) RSSI collection. The
calculated distances are sent to the central nodes for collection. iv)
Attacker detection. Once the distances are calculated they are used by the
central node to detect attacks. To evaluate this approach the authors
implement their algorithm as well as a wormhole attacker in Cooja, using Tmote
Sky nodes, with a non constrained central component running a Linux pc. The
experiments are done on 8, 16, and 24 nodes. Evaluation is performed on energy
overhead as well as true positive detection.

As the tradional grid moved towards a smart grid, several new security
challenges arise, to cater to this, Zhang et al. (2011) (Zhang et al., 2011)
propose an IDS to be able to detect attacks on smart grids. Most of these
changes will occur as an Internet-like communications network is superimposed
on top of the current power grid using wireless technologies including
802.15.4, 802.11, and the Zigbee protocol. Each of these will expose the power
grid to cyber security threats. As the smart grid infrastructure is expansive
and covers several components, including power plants, homes and the transfer
network itself, the authors propose a distributed collaborative approach with
different IDSs at every layer of the smart infrastructure. Firstly the authors
consider a network architecture with three key components, 1) home area
networks (HAN), 2) neighbourhood area networks (NAN), and 3) wide area
networks (WAN). Each of these types networks will contain a different kind of
IDS. At the HAN level the main component is the service module with provides
real time energy costs and consumption to the costumer and record usage
statistics for the grid. The HAN IDS will take the form of a Host Based IDS,
to detect malicious behaviour on the sensing module itself. At the NAN level,
we see a large metering system which combines all the HAN networks in the
area. Each HAN will report to its nearest HAN and all output from the
individual HIDS is sent to the NAN to process. The NAN contains a Network
based IDS to detect network attacks which may come from within the network our
aimed at attacking the metering infrastructure. Finally the WAN, is in charge
of the whole metering infrastructure for the grid, receiving data from all the
NANs, the job of the NAN IDS is to provide a barrier between the outside
network data and the SCADA controllers in the grid to ensure that the energy
and service cooperation can be successful. At the network layer attacks taken
in consideration are DoS attacks, man in the middle attacks and data leakage
of the costumer data. Each of the different IDS will incorporate an Immunity
based algorithm for intrusion detection. The authors specifically choose the
AIRS2Parallel (Watkins and Timmis, 2004) and CLONALG (Brownlee, 2005)
algorithm, and evaluate both approaches. To evaluate their approach the
authors firstly train their algorithms using trace data from the KDD data set,
and secondly they run a simulation in MATLAB, where they construct a
hierarchical architecture with each of their components systems. Each of their
different IDS are tested using the two algorithms and results show
AIRS2Parallel having better results, the only evaluating having been done on
detection and not performance.

Yu and Tsai (2008) (Yu and Tsai, 2008), propose a hybrid approach to intrusion
detection combining both a anomaly based technique and specification based
technique, with the unique approach that the behaviours and specifications are
automatically computed. In their proposed architecture, each node is equipped
with an intrusion detection agent (IDA), however there is no collaboration to
avoid insider attacks. Unlike traditional HIDS where it is possible to analyse
all features in the host in constrained systems it is quite possible for a
device to be wiped, so for their IDA they propose two approaches. The authors
propose a Local Intrusion Detection Components (LIDC) to detect whether a node
is being attacked by other nodes within the network. This is then combined
with a Packet based Intrusion Detection component (PIDC), which monitors
network packets sent within the neighbourhood of the node. Any intrusion alert
is then send to the base station to process using a anomaly based IDS. The
authors then propose a comprehensive list of features that are of interest to
the positive behaviour of a WSN, such as transmission rate and packet drop
rate. Using these features the authors propose the use of SLIPPER (Cohen and
Singer, 1999) a machine learning algorithm which automatically generates rules
out of data. Using the features collected by the nodes in the networks the
central IDS is able to automatically generate rules for the IDS without prior
intervention. Each time an alarm is sent the model is tuned in case of false
alarms, and the IDAs are updated. No evaluation is performed on their
approach.

Buennemeyer et al. (2008), (Buennemeyer et al., 2008), provide a device
profiling methodology able to detect battery exhausting attacks on IoT
devices. The IDS works on WiFi and Bluetooth and makes use of a battery
profiling engine CIDE to view the battery drain. CIDE uses a dynamic threshold
calculation algorithm that is able to detect unusual patters of battery drain
that might correlate to attacks. Each host is equipped with an IDS able to
raise alarms regarding battery consumption, this is combined with the
integration to a Network Based IDS to whom alerts are sent. For their specific
work the authors make use of SNORT, and the alerts are correlated to rules in
the central IDS. To minimise the number of alerts and false positives the
battery thresholds are dynamically tuned during operation. Each device in the
system must be configured with a smart battery whose parameters are tracked by
the CIDE installed on the device. A CIDE will also be present on a server,
this provides a graphical user interface able to let the user configure the
device characteristics as well as monitor the battery fluctuations. To
evaluate the best reporting rate for the IDSs on the device the users
implemented their system on, Ten Dell Axim X51 PDAs, all with the same
settings and initial battery levels. Further experimentation was done on
several other devices and they were able to determine optimal polling rates to
diminish battery rates, with the caveat that in some cases due to high optimal
polling rate it let them vulnerable to timing attacks within the poll
interval. As an additional evaluation the author performed a usability study
with 31 participants, collecting feedback on usability of the tool, quality of
reporting and several other metrics, the expert feedback found their hybrid
approach to be very useful. The authors also showed changes made in accordance
with this usability study.

Moyers et al. (2010) (Moyers et al., 2010), propose Multi-Vector Portable
Intrusion Detection System (MVP-IDS). In this work the authors expand on the
work of B-SIPS by introducing an element of network monitoring to the battery
drain evaluation performed by previous work. They integrate Snort rules within
the HOST IDS to perform more pre-detection at the device level directly. The
MVP-IDS approach is to be able to correlate specific changes in battery usage
to network traffic incoming to the divide level. The architecture of the
system has four components 1) B-SIPS client, for current anomaly triggering,
2) SNORT WiFi module for WiFi attack detection, 3) SNORT Bluetooth attack
detection system for Bluetooth attacks, and 4) CIDE server correlation
response and holistic view of the system. In Buennemeyer’s B-SIPS project,
B-SIPS clients sent device status reports to the CIDE server, but never
received any response because the system was implemented with uni-directional
communication. An improvement that was added in the creation of MVP-IDS was
changing the system to operate in a bi-directional manner. Not only do B-SIPS
clients send device status report to the CIDE server, but the CIDE server also
then replies when action needs to be taken to secure the device and alert the
user of malicious activity. No evaluation was performed on their approach and
no discussion on how their new approach would improve upon the existing IDS.

##  8\. Collecting Tools for Unified Evaluation

To to gather a comprehensive evaluation of the capabilities of IDSs for IoT we
firstly needed to gather implementations for the tools. To do so we firstly
classified the tools into four categories, i) tools which had code available
online, ii) tools which didn’t have code online, however discussed
implementations and therefore should have code, iii) tools which had
implemented in simulations or theoretically analysed, as perhaps they has
since prepared an implementation, and, iv) tools which had no evaluation nor
discussed implementation details.

Secondly we needed to get a updated list of contact details for all the
authors, as some of the papers were quite old and due to the nature of
academic positions moving around a lot, many of the emails on the papers were
outdated. To get up to date emails we first looked up to see if the author had
moved to a new institution and had new contact details, this allowed us to
find several new emails. In the case of the authors leaving academia as was
often the case for PhD and masters students, we looked for Linkedin profiles
for emails, even though this was not always possible. Whilst it was often the
case for professors or senior members of staff to have remained in the same
place, our intuition was that it was worth contacting the full author list as
it is often the case that the implementation is performed by the less senior
members of staff and therefore it is more likely to get access from them. For
some authors we were unable to identify their new email or whether the current
email was still valid. In total 136, email addressed were collected, to the
best of our knowledge with the most up to date details.

The third step was contacting the authors for their tools. We constructed an
email template, customisable for each tool paper, as can be seen in Appendix A
and Appendix B. The email has seven components, 1) a custom greeting to the
specific authors addressing them by their title, 2) a short introduction about
who I am as well as all my contact details, 3) an introduction to the work I
am working on, 3) reasons for contacting them with specific reference to their
paper, 4) a personalised comment on their work, manually generated, 5) request
for the tool code, 6) a query asking how authors would like to be cited in
case of publications and if any licenses applied to their tools and 7) a
closing statement thanking them and letting them know that if they may have
any question they may feel free to contact me.

Emails were sent out, in two stages, the first stage was sent out to any
author in category ii of classification, except for tools that were not
suitable to test in our designed evaluation IoT tesbed, an example of this was
tools that looked at physical intruders in a home. A second stage was sent out
to authors in category iii and iv of classification whose proposed solutions
may reasonably be implemented (and may have been since), this included
approaches who implemented their evaluation through a network simulator. A
total of 84 authors were contacted via email in the two stages.

The resulting collection totalled to, three tools in category iii, whose code
was available on GitHub, of which all three were based on CONTIKI simulator,
and non suitable to our testing environment. One tool pertaining to category
ii, which responded to my emails and provided a code implementation (not
publicly available). A single tool in category i whose implementation was
available online. And no responses in other categories. The net collection
resulted in 4/51 papers with code shared publicly, 1/84 authors who responded
to a request for implementation, and 46/51 papers with no code accounted for.
We do not claim that our inability to find these tools means they were
unavailable but merely that following our protocol, these were the ones
available to us. We therefore relegate the initial goal of evaluating the
available tools under the same environment to a future work.

###  8.1. Analysis

The review we have conducted has shown there is a dire need for better
evaluation of these systems. Whilst several of these approaches make use of
simulation, which one would presume provides more ease in comparing results,
there is still a lack of disclosure of both datasets and the tools themselves.
We observe that this issue is not unique to the area of IoT, as even on
traditional IDS papers, there is a lack of available datasets, with many
evaluations still relying on outdated datasets such as KDD 99, almost two
decades later.

The metrics of evaluation are something that we choose as a core feature of
our approach. Whilst accuracy is very important for any prediction system,
perhaps not as much as one would think. The ability for an IDS to perform,
run, and be analysed is just as if not more important. Consequently,
simulation might not be enough to realistically evaluate the system. There are
of course limitations in terms of constraints of access to realistic test beds
for implementation of the system.

##  9\. Conclusion

In this work we have extensively analysed and summarised the current state of
the art in intrusion detection in the IoT. We have reviewed and summaries 51
proposed tools and technologies and characterised them. Our analysis has shown
several new insights into how research in this field is developing, and
potential gaps for future works. We have identified a major gap for the
reproducibility of the tools evaluation and also discussed shortcomings in the
current evaluation process, namely 1) single focus on detection accuracy; 2)
lack of unified evaluation methodology; and 3) disregard for evaluation
regarding usability of these tools. Our findings indicate that perhaps the
very large numbers of papers in this area is in part due to the inability to
build upon the software and the lack of research into tools that can be
monitored and supervisable. We see a need for a shift to focus away from
development of new techniques to a focus on making these tools that could
scale and be usable across IoT deployments. The analysis shows that this is in
part due to _how_ the tools have been evaluated and we suggest the usage of
metrics from Etalle, S (2019) (Etalle, 2017) as a better set of metrics
against which to decide the suitability of an IDS for IoT systems. As a final
consideration we see a thorough comparative evaluation by these tools through
this evaluation a beneficial and necessary task, although we reserve this for
future work as there currently is not enough access to the proposed tools.

## References

  * (1)
  * Abhishek et al. (2018) Nalam Venkata Abhishek, Teng Joon Lim, Biplab Sikdar, and Anshoo Tandon. 2018.  An intrusion detection system for detecting compromised gateways in clustered IoT networks. In _2018 IEEE International Workshop Technical Committee on Communications Quality and Reliability (CQR)_. IEEE, 1–6.
  * Amaral et al. (2014) João P Amaral, Luís M Oliveira, Joel JPC Rodrigues, Guangjie Han, and Lei Shu. 2014\.  Policy and network-based intrusion detection system for IPv6-enabled wireless sensor networks. In _2014 IEEE International Conference on Communications (ICC)_. IEEE, 1796–1801.
  * Amouri et al. (2018) Amar Amouri, Vishwa T Alaparthy, and Salvatore D Morgera. 2018\.  Cross layer-based intrusion detection based on network behavior for IoT. In _2018 IEEE 19th Wireless and Microwave Technology Conference (WAMICON)_. IEEE, 1–4.
  * Arnaboldi and Morisset (2017) Luca Arnaboldi and Charles Morisset. 2017.  Quantitative analysis of dos attacks and client puzzles in iot systems. In _International Workshop on Security and Trust Management_. Springer, 224–233.
  * Arnaboldi and Morisset (2018a) Luca Arnaboldi and Charles Morisset. 2018a.  Generating Synthetic Data for Real World Detection of DoS Attacks in the IoT. In _Federation of International Conferences on Software Technologies: Applications and Foundations_. Springer, 130–145.
  * Arnaboldi and Morisset (2018b) Luca Arnaboldi and Charles Morisset. 2018b.  LISA: Predicting the Impact of DoS Attacks on Real-World Low Power IoT Systems.  _Foundations of Computer Security Workshop_ (2018).
  * Arnaboldi and Tschofenig (2019) Luca Arnaboldi and Hannes Tschofenig. 2019.  A Formal Model for Delegated Authorization of IoT Devices Using ACE-OAuth. In _OAuth Security Workshop_.
  * Arolkar et al. (2011) Harshal A Arolkar, Shraddha P Sheth, and Vaidehi P Tamhane. 2011.  Ant colony based approach for intrusion detection on cluster heads in WSN.. In _ICCCS_. 523–526.
  * Arrington et al. (2016) Briana Arrington, LiEsa Barnett, Rahmira Rufus, and Albert Esterline. 2016.  Behavioral modeling intrusion detection system (bmids) using internet of things (iot) behavior-based anomaly detection via immunity-inspired algorithms. In _2016 25th International Conference on Computer Communication and Networks (ICCCN)_. IEEE, 1–6.
  * Axelsson (2000) Stefan Axelsson. 2000\.  The base-rate fallacy and the difficulty of intrusion detection.  _ACM Transactions on Information and System Security (TISSEC)_ 3, 3 (2000), 186–205.
  * Besson and Leleu (2009) Lionel Besson and Philippe Leleu. 2009.  A distributed intrusion detection system for ad-hoc wireless sensor networks: the AWISSENET distributed intrusion detection system. In _2009 16th International Conference on Systems, Signals and Image Processing_. IEEE, 1–3.
  * Bostani and Sheikhan (2017) Hamid Bostani and Mansour Sheikhan. 2017.  Hybrid of anomaly-based and specification-based ids for internet of things using unsupervised opf based on mapreduce approach.  _Computer Communications_ 98 (2017), 52–71.
  * Brownlee (2005) Jason Brownlee. 2005\.  Clonal selection theory & clonalg-the clonal selection classification algorithm (csca).  _Swinburne University of Technology_ (2005), 38.
  * Buennemeyer et al. (2007) Timothy K Buennemeyer, Michael Gora, Randy C Marchany, and Joseph G Tront. 2007.  Battery exhaustion attack detection with small handheld mobile computers. In _Portable Information Devices_.
  * Buennemeyer et al. (2008) Timothy K Buennemeyer, Theresa M Nelson, Lee M Clagett, John P Dunning, Randy C Marchany, and Joseph G Tront. 2008.  Mobile device profiling and intrusion detection using smart batteries. In _Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008)_. IEEE, 296–296.
  * Butun et al. (2014) Ismail Butun, Salvatore D Morgera, and Ravi Sankar. 2014\.  A survey of intrusion detection systems in wireless sensor networks.  _IEEE communications surveys & tutorials_ 16, 1 (2014), 266–282.
  * Castro et al. (2002) Leandro Nunes Castro, Leandro Nunes De Castro, and Jonathan Timmis. 2002.  _Artificial immune systems: a new computational intelligence approach_.  Springer Science & Business Media.
  * Cervantes et al. (2015) Christian Cervantes, Diego Poplade, Michele Nogueira, and Aldri Santos. 2015.  Detection of sinkhole attacks for supporting secure routing on 6LoWPAN for Internet of Things. In _2015 IFIP/IEEE International Symposium on Integrated Network Management (IM)_. IEEE, 606–611.
  * Cho et al. (2009) Eung Jun Cho, Jin Ho Kim, and Choong Seon Hong. 2009.  Attack model and detection scheme for botnet on 6LoWPAN. In _Asia-Pacific Network Operations and Management Symposium_. Springer, 515–518.
  * Chormunge and Jena (2015) Smita Chormunge and Sudarson Jena. 2015.  Efficiency and Effectiveness of Clustering Algorithms for High Dimensional Data.  _International Journal of Computer Applications_ 125, 11 (2015).
  * Cohen and Singer (1999) William W Cohen and Yoram Singer. 1999.  A simple, fast, and effective rule learner.  _AAAI/IAAI_ 99 (1999), 335–342.
  * Coppolino et al. (2013) Luigi Coppolino, Salvatore DAntonio, Alessia Garofalo, and Luigi Romano. 2013.  Applying data mining techniques to intrusion detection in wireless sensor networks. In _2013 Eighth International Conference on P2P, Parallel, Grid, Cloud and Internet Computing_. IEEE, 247–254.
  * Craenen and Eiben (2002) B Craenen and A Eiben. 2002\.  Computational intelligence. Encyclopedia of Life Support Sciences.  _EOLSS, EOLSS Co. Ltd_ (2002).
  * Danda and Hota (2016) Jagan Mohan Reddy Danda and Chittaranjan Hota. 2016.  Attack identification framework for IoT devices.  In _Information Systems Design and Intelligent Applications_. Springer, 505–513.
  * Deng et al. (2018) Lianbing Deng, Daming Li, Xiang Yao, David Cox, and Haoxiang Wang. 2018.  Mobile network intrusion detection for IoT system based on transfer learning algorithm.  _Cluster Computing_ (2018), 1–16.
  * Di Pietro and Mancini (2008) Roberto Di Pietro and Luigi V Mancini. 2008.  _Intrusion detection systems_. Vol. 38.  Springer Science & Business Media.
  * Echeverria et al. (2019) Sebastian Echeverria, Ludwig Seitz, Dan Klinedinst, and Grace Lewis. 2019.  _ACE Clients in Disadvantaged Networks_.  Internet-Draft draft-secheverria-ace-client-disadvantaged-00. Internet Engineering Task Force.  https://datatracker.ietf.org/doc/html/draft-secheverria-ace-client-disadvantaged-00 Work in Progress.
  * Elrawy et al. (2018) Mohamed Faisal Elrawy, Ali Ismail Awad, and Hesham FA Hamed. 2018\.  Intrusion detection systems for IoT-based smart environments: a survey.  _Journal of Cloud Computing_ 7, 1 (2018), 21\.
  * Esquivel-Vargas et al. (2017) Herson Esquivel-Vargas, Marco Caselli, and Andreas Peter. 2017\.  Automatic deployment of specification-based intrusion detection in the BACnet protocol. In _Proceedings of the 2017 Workshop on Cyber-Physical Systems Security and PrivaCy_. 25–36.
  * Etalle (2017) Sandro Etalle. 2017\.  From intrusion detection to software design. In _European Symposium on Research in Computer Security_. Springer, 1–10.
  * Etalle (2019) Sandro Etalle. 2019\.  Network Monitoring of Industrial Control Systems: The Lessons of SecurityMatters. In _Proceedings of the ACM Workshop on Cyber-Physical Systems Security & Privacy_. 1–1.
  * Fu et al. (2017) Yulong Fu, Zheng Yan, Jin Cao, Ousmane Koné, and Xuefei Cao. 2017.  An automata based intrusion detection method for internet of things.  _Mobile Information Systems_ 2017 (2017).
  * Garcia-Font et al. (2017) Victor Garcia-Font, Carles Garrigues, and Helena Rifà-Pous. 2017\.  Attack classification schema for smart city WSNs.  _Sensors_ 17, 4 (2017), 771.
  * Garcia-Teodoro et al. (2009) Pedro Garcia-Teodoro, Jesus Diaz-Verdejo, Gabriel Maciá-Fernández, and Enrique Vázquez. 2009\.  Anomaly-based network intrusion detection: Techniques, systems and challenges.  _computers & security_ 28, 1-2 (2009), 18–28.
  * Gupta et al. (2013) Abhishek Gupta, Om Jee Pandey, Mahendra Shukla, Anjali Dadhich, Samar Mathur, and Anup Ingle. 2013\.  Computational intelligence based intrusion detection systems for wireless communication and pervasive computing networks. In _2013 IEEE International Conference on Computational Intelligence and Computing Research_. IEEE, 1–7.
  * Haataja (2008) Keijo MJ Haataja. 2008\.  New efficient intrusion detection and prevention system for Bluetooth networks. In _Proceedings of the 1st international conference on MOBILe Wireless MiddleWARE, Operating Systems, and Applications_. ICST (Institute for Computer Sciences, Social-Informatics and …, 16.
  * Hadžiosmanović et al. (2014) Dina Hadžiosmanović, Robin Sommer, Emmanuele Zambon, and Pieter H Hartel. 2014.  Through the eye of the PLC: semantic security monitoring for industrial processes. In _Proceedings of the 30th Annual Computer Security Applications Conference_. ACM, 126–135.
  * Han et al. (2013) Guangjie Han, Jinfang Jiang, Wen Shen, Lei Shu, and Joel Rodrigues. 2013.  IDSEP: a novel intrusion detection scheme based on energy prediction in cluster-based wireless sensor networks.  _IET Information Security_ 7, 2 (2013), 97–105.
  * Hassanzadeh and Stoleru (2011) Amin Hassanzadeh and Radu Stoleru. 2011.  Towards optimal monitoring in cooperative ids for resource constrained wireless networks. In _Computer Communications and Networks (ICCCN), 2011 Proceedings of 20th International Conference on_. IEEE, 1–8.
  * Hui Suo Jiafu Wan (2014) Caifeng Zou Jianqi Liu Hui Suo Jiafu Wan. 2014\.  Security in the Internet of Things: A Review.  _International Journal of Computer Applications_ (2014).
  * Iera and Morabito (2010) Atzori Luigi Antonio Iera and Giacomo Morabito. 2010.  The Internet of Things: A Survey.  _Computer Networks_ (2010).
  * Jiang et al. (2012) Tingyao Jiang, Gangliang Wang, and Heng Yu. 2012.  A dynamic intrusion detection scheme for cluster-based wireless sensor networks. In _World Automation Congress 2012_. IEEE, 259–261.
  * Kai Zhao (2013) Lina Ge Kai Zhao. 2013\.  A Survey on the Internet of Things Security.  _Computational Intelligence and Security (CIS)_ (2013).
  * Kasinathan et al. (2013a) Prabhakaran Kasinathan, Gianfranco Costamagna, Hussein Khaleel, Claudio Pastrone, and Maurizio A Spirito. 2013a.  An ids framework for internet of things empowered by 6lowpan. In _Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security_. ACM, 1337–1340.
  * Kasinathan et al. (2013b) Prabhakaran Kasinathan, Claudio Pastrone, Maurizio A Spirito, and Mark Vinkovits. 2013b.  Denial-of-Service detection in 6LoWPAN based Internet of Things. In _2013 IEEE 9th international conference on wireless and mobile computing, networking and communications (WiMob)_. IEEE, 600–607.
  * Khan and Herrmann (2017) Zeeshan Ali Khan and Peter Herrmann. 2017.  A trust based distributed intrusion detection mechanism for internet of things. In _2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA)_. IEEE, 1169–1176.
  * Kim (2015) Sang Wu Kim. 2015\.  Physical integrity check in cooperative relay communications.  _IEEE Transactions on Wireless Communications_ 14, 11 (2015), 6401–6413.
  * Kruegel and Vigna (2003) Christopher Kruegel and Giovanni Vigna. 2003.  Anomaly detection of web-based attacks. In _Proceedings of the 10th ACM conference on Computer and communications security_. ACM, 251–261.
  * Le et al. (2016) Anhtuan Le, Jonathan Loo, Kok Chai, and Mahdi Aiash. 2016\.  A specification-based IDS for detecting attacks on RPL-based network topology.  _Information_ 7, 2 (2016), 25.
  * Le et al. (2011) Anhtuan Le, Jonathan Loo, Yuan Luo, and Aboubaker Lasebae. 2011\.  Specification-based IDS for securing RPL from topology attacks. In _2011 IFIP Wireless Days (WD)_. IEEE, 1–3.
  * Lee et al. (2014) Tsung-Han Lee, Chih-Hao Wen, Lin-Huang Chang, Hung-Shiou Chiang, and Ming-Chun Hsieh. 2014.  A lightweight intrusion detection scheme based on energy consumption analysis in 6LowPAN.  In _Advanced Technologies, Embedded and Multimedia for Human-centric Computing_. Springer, 1205–1213.
  * Lin et al. (2015) Wei-Chao Lin, Shih-Wen Ke, and Chih-Fong Tsai. 2015\.  CANN: An intrusion detection system based on combining cluster centers and nearest neighbors.  _Knowledge-based systems_ 78 (2015), 13–21.
  * Liu et al. (2011) Caiming Liu, Jin Yang, Run Chen, Yan Zhang, and Jinquan Zeng. 2011.  Research on immunity-based intrusion detection technology for the internet of things. In _2011 Seventh International Conference on Natural Computation_ , Vol. 1. IEEE, 212–216.
  * Liu et al. (2018) Liqun Liu, Bing Xu, Xiaoping Zhang, and Xianjun Wu. 2018\.  An intrusion detection method for internet of things based on suppressed fuzzy clustering.  _EURASIP Journal on Wireless Communications and Networking_ 2018, 1 (2018), 113.
  * Liu and Yu (2008) Yang Liu and Fengqi Yu. 2008.  Immunity-based intrusion detection for wireless sensor networks. In _2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence)_. IEEE, 439–444.
  * Lundin and Jonsson (2000) Emilie Lundin and Erland Jonsson. 2000.  Anomaly-based intrusion detection: privacy concerns and other problems.  _Computer networks_ 34, 4 (2000), 623–640.
  * Luo and Nagarajan (2018) Tie Luo and Sai G Nagarajan. 2018.  Distributed anomaly detection using autoencoder neural networks in wsn for iot. In _2018 IEEE International Conference on Communications (ICC)_. IEEE, 1–6.
  * Matsunaga et al. (2014) Takumi Matsunaga, Kentaroh Toyoda, and Iwao Sasase. 2014\.  Low false alarm rate RPL network monitoring system by considering timing inconstancy between the rank measurements. In _2014 11th International Symposium on Wireless Communications Systems (ISWCS)_. IEEE, 427–431.
  * Mell et al. (2003) Peter Mell, Vincent Hu, Richard Lippmann, Josh Haines, and Marc Zissman. 2003.  An overview of issues in testing intrusion detection systems.
  * Midi et al. (2017) Daniele Midi, Antonino Rullo, Anand Mudgerikar, and Elisa Bertino. 2017.  Kalis—A system for knowledge-driven adaptable intrusion detection for the Internet of Things. In _2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)_. IEEE, 656–666.
  * Milenkoski et al. (2015) Aleksandar Milenkoski, Marco Vieira, Samuel Kounev, Alberto Avritzer, and Bryan D Payne. 2015\.  Evaluating computer intrusion detection systems: A survey of common practices.  _ACM Computing Surveys (CSUR)_ 48, 1 (2015), 12\.
  * Minakov et al. (2016) Ivan Minakov, Roberto Passerone, Alessandra Rizzardi, and Sabrina Sicari. 2016.  A comparative study of recent wireless sensor network simulators.  _ACM Transactions on Sensor Networks (TOSN)_ 12, 3 (2016), 1–39.
  * Misra et al. (2011) Sudip Misra, P Venkata Krishna, Harshit Agarwal, Antriksh Saxena, and Mohammad S Obaidat. 2011\.  A learning automata based solution for preventing distributed denial of service in Internet of things. In _2011 International Conference on Internet of Things and 4th International Conference on Cyber, Physical and Social Computing_. IEEE, 114–122.
  * Moyers et al. (2010) Benjamin R Moyers, John P Dunning, Randolph C Marchany, and Joseph G Tront. 2010.  The multi-vector portable intrusion detection system (MVP-IDS): a hybrid approach to intrusion detection for portable information devices. In _2010 IEEE International Conference on Wireless Information Technology and Systems_. IEEE, 1–4.
  * OConnor and Reeves (2008) Terrence OConnor and Douglas Reeves. 2008.  Bluetooth network-based misuse detection. In _2008 Annual Computer Security Applications Conference (ACSAC)_. IEEE, 377–391.
  * Oh et al. (2014) Doohwan Oh, Deokho Kim, and Won Ro. 2014.  A malicious pattern detection engine for embedded security systems in the Internet of Things.  _Sensors_ 14, 12 (2014), 24188–24211.
  * Patcha and Park (2007) Animesh Patcha and Jung-Min Park. 2007.  An overview of anomaly detection techniques: Existing solutions and latest technological trends.  _Computer networks_ 51, 12 (2007), 3448–3470.
  * Patel (2014) J. Sathish Kumar Dhiren R. Patel. 2014\.  A Survey on Internet of Things: Security and Privacy Issues.  _International Journal of Computer Applications_ (2014).
  * Pongle and Chavan (2015) Pavan Pongle and Gurunath Chavan. 2015.  Real time intrusion and wormhole attack detection in internet of things.  _International Journal of Computer Applications_ 121, 9 (2015).
  * Raza et al. (2013) Shahid Raza, Linus Wallgren, and Thiemo Voigt. 2013\.  SVELTE: Real-time intrusion detection in the Internet of Things.  _Ad hoc networks_ 11, 8 (2013), 2661–2674.
  * Sabhnani and Serpen (2004) Maheshkumar Sabhnani and Gursel Serpen. 2004.  Why machine learning algorithms fail in misuse detection on KDD intrusion detection data set.  _Intelligent data analysis_ 8, 4 (2004), 403–415.
  * Scarfone and Mell (2007) Karen Scarfone and Peter Mell. 2007.  Guide to intrusion detection and prevention systems (idps).  _NIST special publication_ 800, 2007 (2007), 94\.
  * Sedjelmaci and Senouci (2013) Hichem Sedjelmaci and Sidi Mohammed Senouci. 2013.  Efficient and lightweight intrusion detection based on nodes’ behaviors in wireless sensor networks. In _Global Information Infrastructure Symposium-GIIS 2013_. IEEE, 1–6.
  * Sedjelmaci et al. (2016) Hichem Sedjelmaci, Sidi Mohammed Senouci, and Mohamad Al-Bahri. 2016.  A lightweight anomaly detection technique for low-resource IoT devices: A game-theoretic methodology. In _2016 IEEE International Conference on Communications (ICC)_. IEEE, 1–6.
  * Shreenivas et al. (2017) Dharmini Shreenivas, Shahid Raza, and Thiemo Voigt. 2017\.  Intrusion Detection in the RPL-connected 6LoWPAN Networks. In _Proceedings of the 3rd ACM International Workshop on IoT Privacy, Trust, and Security_. ACM, 31–38.
  * Sommer and Paxson (2010) Robin Sommer and Vern Paxson. 2010.  Outside the closed world: On using machine learning for network intrusion detection. In _2010 IEEE symposium on security and privacy_. IEEE, 305–316.
  * Song et al. (2010) Xianfeng Song, Guangxi Chen, and Xiaolong Li. 2010.  A Weak Hidden Markov Model based intrusion detection method for wireless sensor networks. In _2010 International Conference on Intelligent Computing and Integrated Systems_. IEEE, 887–889.
  * Stelte and Rodosek (2013) Björn Stelte and Gabi Dreo Rodosek. 2013.  Thwarting attacks on zigbee-removal of the killerbee stinger. In _Proceedings of the 9th International Conference on Network and Service Management (CNSM 2013)_. IEEE, 219–226.
  * Summerville et al. (2015) Douglas H Summerville, Kenneth M Zach, and Yu Chen. 2015\.  Ultra-lightweight deep packet anomaly detection for internet of things devices. In _2015 IEEE 34th International Performance Computing and Communications Conference (IPCCC)_. IEEE, 1–8.
  * Tan et al. (2002) Kymie MC Tan, Kevin S Killourhy, and Roy A Maxion. 2002\.  Undermining an anomaly-based intrusion detection system using common exploits. In _International Workshop on Recent Advances in Intrusion Detection_. Springer, 54–73.
  * Thanigaivelan et al. (2016) Nanda Kumar Thanigaivelan, Ethiopia Nigussie, Rajeev Kumar Kanth, Seppo Virtanen, and Jouni Isoaho. 2016\.  Distributed internal anomaly detection system for internet-of-things. In _2016 13th IEEE Annual Consumer Communications & Networking Conference (CCNC)_. IEEE, 319–320.
  * Vora et al. (2013) Pritesh Vora, Bhavesh Oza, et al. 2013\.  A survey on k-mean clustering and particle swarm optimization.  _International Journal of Science and Modern Engineering_ 1, 3 (2013), 1–14.
  * Wallgren et al. (2013) Linus Wallgren, Shahid Raza, and Thiemo Voigt. 2013\.  Routing Attacks and Countermeasures in the RPL-based Internet of Things.  _International Journal of Distributed Sensor Networks_ 9, 8 (2013), 794326\.
  * Wang and Megalooikonomou (2005) Qiang Wang and Vasileios Megalooikonomou. 2005.  A clustering algorithm for intrusion detection. In _Data Mining, Intrusion Detection, Information Assurance, and Data Networks Security 2005_ , Vol. 5812. International Society for Optics and Photonics, 31–39.
  * Watkins and Timmis (2004) Andrew Watkins and Jon Timmis. 2004.  Exploiting parallelism inherent in AIRS, an artificial immune classifier. In _International Conference on Artificial Immune Systems_. Springer, 427–438.
  * Weaver et al. (2013) Randy Weaver, Dawn Weaver, and Dean Farwood. 2013\.  _Guide to network defense and countermeasures_.  Cengage Learning.
  * Wendt and Potkonjak (2015) Teng Xu James B Wendt and Miodrag Potkonjak. 2015.  Security of IoT Systems: Design Challenges and Opportunities.  _Informational System Frontiers_ (2015).
  * Wu and Banzhaf (2010) Shelly Xiaonan Wu and Wolfgang Banzhaf. 2010.  The use of computational intelligence in intrusion detection systems: A review.  _Applied soft computing_ 10, 1 (2010), 1–35.
  * Yadav and Srinivasan (2010) Kuldeep Yadav and Avinash Srinivasan. 2010.  iTrust: an integrated trust framework for wireless sensor networks. In _Proceedings of the 2010 ACM Symposium on Applied Computing_. ACM, 1466–1471.
  * Yu and Tsai (2008) Zhenwei Yu and Jeffrey JP Tsai. 2008.  A framework of machine learning based intrusion detection for wireless sensor networks. In _2008 IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (sutc 2008)_. IEEE, 272–279.
  * Zarpelao et al. (2017) Bruno Bogaz Zarpelao, Rodrigo Sanches Miani, Cláudio Toshio Kawakani, and Sean Carlisto de Alvarenga. 2017\.  A survey of intrusion detection in Internet of Things.  _Journal of Network and Computer Applications_ 84 (2017), 25–37.
  * Zhang et al. (2011) Yichi Zhang, Lingfeng Wang, Weiqing Sun, Robert C Green, and Mansoor Alam. 2011.  Artificial immune system based intrusion detection in a distributed hierarchical network architecture of smart grid. In _2011 IEEE Power and Energy Society General Meeting_. IEEE, 1–8.
  * Zurada et al. (1995) JM Zurada, RJ Marks, and J Robinson. 1995.  Review of computational intelligence: imitating life.

##  Appendix A Survey IDS Request Template Letter

Dear ¡INSERT AUTHOR LIST¿,

I am a PhD Student working on IoT Security at Newcastle University (MyInfo)
and I am currently working on developing an IoT security testbed. My work
hopes to test out various security solutions in a range of test IoT
environments in the form of embedded virtual machines hosted in various
network configurations. I am contacting you in regards to your work ¡PAPER
TITLE HERE¿. ¡INSERT UNIQUE COMMENT ON PAPER¿. I was hoping you could provide
me with an implementation of your work to use as I was unable to find a
publicly available repository?

Please let me know how you would prefer I cite your work in the case of
publication and what license restrictions might be applicable.

If you have any questions about my work or how your implementation will be
used please don’t hesitate to ask.

Yours sincerely,
Luca Arnaboldi

##  Appendix B Survey IDS Request Example Letter

Dear REDACTED,

I am a PhD Student working on IoT Security at Newcastle University (MyInfo)
and I am currently working on developing an IoT security testbed. My work
hopes to test out various security solutions in a range of test IoT
environments in the form of embedded virtual machines hosted in various
network configurations. I am contacting you in regards to your work REDACTED.
Your presented detection methodology is very powerful showing some excellent
accuracy with low false positives, I would be interested in seeing how well it
would scale in larger systems and different configurations. I was hoping you
could provide me with an implementation of your work to use as I was unable to
find a publicly available repository?

Please let me know how you would prefer I cite your work in the case of
publication and what license restrictions might be applicable.

If you have any questions about my work or how your implementation will be
used please don’t hesitate to ask.

Yours sincerely,
Luca Arnaboldi