##  1 Introduction

Increasing cyber attacks and data breaches require new ways of predicting
attacks, especially previously unseen ones. A very popular tool for
prediction, is a honeypot (Barak, 2020) that aims to lure attackers and learn
their tactics. While honeypots have known advantages such as ease of
installation and low resource usage (Campbell et al., 2015), there exist
drawbacks, such as limited vision, discovery and fingerprinting, that increase
the risk of takeover (Mokube and Adams, 2007). Thus, honeypots are often
deployed in conjunction with Network Anomaly Detection Systems (NADS) (Baykara
and Das, 2018; Kondra et al., 2016).

NADS are capable of detecting new attacks, a key requirement in cyber
security. However, they suffer from high false positive rates. Furthermore,
honeypot aided NADS are generally deployed in public domain networks. The use
of honeypots in Local Area Networks (LANs) has not been widely explored. In
this paper, we propose a novel framework, Honeyboost, that enhances LAN
honeypot performance with Network Anomaly Detection (NAD), while giving low
false positive rates. Honeyboost identifies most anomalous nodes in the LAN,
before they even access the honeypot, enabling better prediction of cyber
attacks.

While many researchers have studied honeypots and anomaly detection, most
place these detection tools in the public/global IP domain or in Internet
gateway routers. This results in efficient detection of global cyber-attack
behaviors such as global-scans, Distributed Denial of Service (DDoS) attacks,
or botnet constructions. However, such placing is inadequate for detecting
local or LAN internal cyber-attack behaviors such as malware propagation,
insider attacks, or data stealing through direct communications in the same
network segment without passing through routers. This may happen when (1) a
malware is pre-installed in a mobile device that connects to a WiFi, or (2) a
malware is delivered to a computer through a phishing e-mail or Social Network
Sites (SNS).

This paper focuses on honeypot traffic in a local area network, where the
behavior of the address resolution protocol (ARP) can be observed. ARP
requests are normally broadcast in the local network segment in order to find
the media access control (MAC) address of the target IP address before
forwarding an IP packet to the host. The sequences of ARP requests, along with
other protocol data are used as input for anomaly detection. This allows
suspicious behavior, such as multiple attempts to access many IP addresses in
the local area network, to be detected as anomalous. This LAN based honeypot
research is unique as it allows us to predict anomalous nodes, something that
cannot happen in a global setting as the nodes wouldn’t be known.

We recognize that not all anomalies – nodes that access the honeypot – are
malicious. In general, anomalies are mainly caused by  1. malware trying to
intrude into other hosts or steal data from the network data storage/camera,
2. security software equipped with features of basic vulnerability testing,
and  3. network operators intentionally accessing computers in the LAN to
check their status.

Honeyboost treats the output traffic of each node/host in a LAN as a time
series, and computes node-level features describing the behavior of nodes over
time. To the best of our knowledge, this is the first study that computes
node-level features by treating the output traffic of each node as a time
series, in honeypot aided NADS. We use an unsupervised anomaly detection (AD)
method called Lookout that uses extreme value theory to detect anomalies and
effective in minimizing false positives. Thus, the performance of the honeypot
is enhanced by using a node-level formulation in conjunction with an AD method
with low false positives.

Identifying anomalous nodes in a network can be formulated as a time series
problem as nodes transmit network packets using multiple protocols at
different time points (Figure 1). Thus, identifying anomalous time series
would yield the nodes that behave quite differently from the rest.

Figure 1: An example of the output of a single node, which can be treated as a
time series. As each protocol gives rise to a different number of features,
this is a varying-dimensional time series.

###  1.1 Challenges

Formulating Network Anomaly Detection (NAD) as a time series problem, whether
it is honeypot aided or not, presents several challenges:

  1. 1.

Varying dimensional time series: Univariate and multivariate time series are
widely studied in the literature from many inter-disciplinary contexts. A
univariate time series is a sequence of real valued numbers indexed in time,
denoted by $\\{u_{1},u_{2},\ldots,u_{t},\ldots\\}$, where $u_{t}\in\mathbb{R}$
for all $t$. A multivariate time series is a sequence of vectors indexed in
time denoted by $\\{\bm{u}_{1},\bm{u}_{2},\ldots,\bm{u}_{t},\ldots\\}$, where
$\bm{u}_{t}\in\mathbb{R}^{p}$ for all $t$. However, here we have a different
situation. As the same node emits packets using different protocols, we have a
time series
$\\{\bm{u}_{t_{1}},\bm{v}_{t_{2}},\bm{w}_{t_{3}},\ldots,\bm{u}_{t_{i}},\bm{v}_{t_{j}},\bm{w}_{t_{k}},\ldots\\}$,
where $\bm{u}_{t_{i}}\in\mathbb{R}^{p}$, $\bm{v}_{t_{j}}\in\mathbb{R}^{q}$ and
$\bm{w}_{t_{k}}\in\mathbb{R}^{m}$ for different values of $p,q$, $m$ and
different times $t_{i}$, $t_{j}$ and $t_{k}$. We call such a time series
varying-dimensional (VD). There has not been much research conducted on VD
time series. A literature search has not indicated any study of anomaly
detection using VD time series.

  2. 2.

Irregularity: Unevenly spaced time series are referred to as irregular time
series. The traditional time series theory is applicable to regular, evenly
spaced time series, that is, a series $\\{u_{1},u_{2},\ldots,u_{t},\ldots\\}$,
where the time between successive entries $u_{i}$ and $u_{i+1}$ is constant.
Here we have an irregular varying-dimensional time series as the nodes
communicate with each other at different time points. As such, when we
consider the VD time series
$\\{\bm{u}_{t_{1}},\bm{v}_{t_{2}},\bm{w}_{t_{3}},\ldots,\bm{u}_{t_{n}},\ldots\\}$,
in general $t_{n+1}-t_{n}\neq t_{n}-t_{n-1}$.

  3. 3.

Sparse literature: The literature on NADS mostly considers a packet based
feature approach. We see this in most publicly available datasets. For example
popular datasets such as KDD Cup 99 and Kyoto dataset contain packet level
observations without source details. While we appreciate that source addresses
may compromise privacy, without even a dummy identifier it is not possible to
construct a time series at the node level. As such, we observe that while
these datasets have been instrumental for the research growth in NADS, they
have also steered the progress towards a packet based feature approach.

While a node-based VD time series approach to NAD has challenges, it takes a
more holistic view on NAD. As such, we expect to gain deeper insights about
network anomalies by using this modeling paradigm.

###  1.2 Our contribution

In this paper, we investigate the problem of honeypot aided NAD as a time
series analysis problem and present Honeyboost – a hybrid framework to find
anomalous nodes. As shown in Figure 2, Honeyboost includes the following
approaches:

  1. 1.

Horizontal approach: We treat the data from each node as a VD time series and
find anomalous time series using a feature based approach, where features are
computed from the node-based VD time series.

  2. 2.

Vertical approach: We focus on each protocol and find anomalous nodes with
respect to each protocol using features relevant to that protocol. We then
amalgamate the results enabling visual interpretation.

Our results demonstrate the following benefits:

  1. 1.

We identify most anomalous nodes before they access the honeypot.

  2. 2.

We compare Honeyboost results obtained by the AD method Lookout with a One
Class Support Vector Machine (OCSVM) and find the false positive rate of
Lookout is much lower compared to the OCSVM.

  3. 3.

We rank anomalies, thus enabling prioritization.

  4. 4.

We gain deeper insights about the anomalous nodes.

  5. 5.

We identify anomalous nodes with suspicious behavior that do not access the
honeypot.

Figure 2: Honeyboost framework comprising horizontal and vertical approaches.
The horizontal approach considers node-level VD time series and identifies
anomalous nodes. The vertical approach considers each protocol separately and
finds anomalies in each protocol space.

The remainder of the paper is organized as follows; In Section 2 we give a
brief introduction to current research in honeypots, anomaly detection and
extreme value theory. The datasets for this study, discussed in Section 3 are
obtained from the Lan-Security Monitoring Project (Ochiai, 2020). The window-
based approach, and the horizontal and vertical methodologies are discussed in
Section 4. We present the early detection results, false positive rates and
amalgamation of horizontal and vertical anomalies in Section 5. We glean
insights from these results in Section 6. Finally, we present our conclusions
in Section 7.

##  2 Background

In this section we provide the current research related to Honeyboost,
including Honeypots, Anomaly detection and Extreme Value Theory (EVT). We
start with honeypot research.

###  2.1 Honeypots

Honeypot-based cyber security research can be categorized into two broad
areas:  1. honeypot architecture design (Fan et al., 2019; Sadasivam and Hota,
2015) and  2. threat detection and prevention.  These two topics are not
mutually exclusive, with some studies encompassing both topics. For example
Jasek et al. (2013) use Honeyfarms, a centralized collection of honeypots and
analysis tools, to analyze and detect advanced persistent threats. In this
section we explore some of the work done on threat detection and prevention
using honeypots.

Zhan et al. (2013) characterize honeypot captured cyber attacks using a
statistical framework. They show that these cyber attacks exhibit long-range
dependence, i.e. the rate of autocorrelation decay is slower than exponential
decay. They use statistical techniques to predict the attack rate, i.e. the
number of attacks per time unit. Almohannadi et al. (2018) use honeypot log
data to evaluate a new threat intelligence technique and find attack patterns.
They admit that the large amount of data produced by honeypots is difficult to
analyze using general purpose techniques, and instead, use an open search
analytics engine called elastisearch. La et al. (2016) present a game-
theoretic model of deception comprising attackers and defenders, using
Bayesian techniques, and verifying their model using simulated data. As part
of the game play, defenders have access to the honeypots. Moore and Al-Nemrat
(2015) review production honeypots and analyze data collected from honeypots
over a period of time. They explore geographical locations of attacks, IP
addresses and ports in this data. Shrivastava et al. (2019) use honeypots to
capture attacks on IoT devices and classify them using supervised machine
learning algorithms.

Honeypots are also used to combat targeted attacks. Denial of Service (DoS)
attacks can cripple entire networks without finding loopholes in security.
Anirudh et al. (2017) discuss a honeypot model for mitigating DoS attacks for
an Internet of Things (IoT) network. Tiruvakadu and Pallapa (2018) discuss
wormhole attacks in the context of mobile, ad-hoc networks and propose a
honeypot based solution to confirm these attacks. A wormhole attack consists
of two or more attackers strategically placing themselves in a network and
creating a tunnel between them. This reduces the shortest path between certain
nodes, inducing legitimate traffic to go through the attacker’s tunnel. They
argue the necessity of an attack confirmation system and use a honeypot to
confirm the attacks using a wormhole attack tree. Zhang et al. (2021) use
honeypots to validate their adversary detection model in an IoT network.
Cybereason, a cyber security technology company, launched a network honeypot
in early 2020 to learn the tactics, techniques and procedures of cyber
criminals. Barak (2020) discusses the lessons learnt from this experiment and
the role of honeypots in critical infrastructure system security. Handa et al.
(2021) discuss the use of honeypots as a tool to obtain standard cyber defense
for small and medium scale businesses. In their book, they describe different
cyber security solutions accessible to small enterprises.

Several recent reviews on honeypots are testament to their rising popularity.
Razali et al. (2019) discuss the importance and history of honeypots in
information security and review their use in IoT networks. Seungjin et al.
(2020) survey honeypot based botnet detection and Matin and Rahardjo (2020)
review malware detection using honeypots.

###  2.2 Anomaly detection

As per Hawkins (1980) “an anomaly/outlier is an observation which deviates so
much from other observations as to arouse suspicion it was generated by a
different mechanism”. Motivated by this well accepted definition we consider
anomalies to be rare observations different from the rest of the points in
some feature space.

Anomaly detection (AD) is an extensively researched topic in many inter-
disciplinary research fields. In computer networks and security research, AD
enables detection of unusual patterns in data that may signify new attacks.
While signature based intrusion detection methods reduce the number of false
positives, they are ineffective in capturing new attacks. AD methods are
commonly used to fill this gap.

Based on the learning paradigm, anomaly detection methodologies can be broadly
categorized into three groups (Goldstein and Uchida, 2016):

  1. 1.

Supervised anomaly detection: A model is trained on labeled data, which
includes anomalies, and then tested on new data.

  2. 2.

Semi-supervised anomaly detection: Two different definitions exist for semi-
supervised anomaly detection.  (a) A model is trained on data including both
labeled and unlabeled instances (Ruff et al., 2020). Usually the number of
labeled data points is much less than the unlabeled ones.  (b) A model is
trained on data that does not include anomalies (Goldstein and Uchida, 2016).
In both scenarios, the resulting model is tested on new data.

  3. 3.

Unsupervised anomaly detection: These techniques do not use a separate set of
labeled data to train the model.

In this study, we focus on unsupervised anomaly detection, as unsupervised
methods are better suited to identify new attacks.

In addition, anomaly detection methods are also categorized as static or
dynamic, depending on whether they evolve with streaming data. The AD
literature encompasses a diverse set of methodologies including density
estimation and probabilistic models, distance-based models, one class
classification models, reconstruction and deep learning models, and cluster-
based and graph-based models. For a general survey of AD methodology, see Wang
et al. (2019) and for a review of deep learning methodologies refer to Ruff et
al. (2020).

####  2.2.1 Anomaly detection in computer networks

Figure 3 shows the generic architecture of an NADS (Ahmed et al., 2016), which
includes pre-processing, anomaly detection, output and evaluation modules. The
preprocessing module performs feature extraction and feature selection at a
packet level (Moustafa et al., 2019). Using either the packet header
information alone or packet header along with payload information, a variety
of features are computed, then used by the Anomaly Detection (AD) module. The
output of the AD module can be real-valued anomaly scores or binary labels
signifying whether a data point is an anomaly or not. The evaluation module is
the pre-action phase, where decisions are made based on the anomaly
scores/labels. Each module can have several sub-modules feeding into other
modules.

Figure 3: The generic architecture of a NADS as depicted in Ahmed et al.
(2016).

Researchers have used anomaly detection methods to identify attacks and
intrusions in computer networks for more than 30 years (Maxion, 1990).
Furthermore, new anomaly detection methods aimed at specific threats and
technologies such as IoT networks are continuously being developed (Naveed and
Wu, 2020). A number of comprehensive reviews on network anomaly detection are
available (Baddar et al., 2014; Ahmed et al., 2016; Fernandes et al., 2019;
Moustafa et al., 2019). We briefly discuss the review by Moustafa et al.
(2019) and touch upon some latest developments in the area.

Moustafa et al. (2019) emphasizes that unlike signature-based methods, anomaly
detection is better suited to capture new attacks. NADS operate by creating a
normal profile and identifying deviations from it. In the pre-processing
stage, features are extracted from the raw data, then reduced in number by
discarding noisy, unimportant features, leaving a smaller set of meaningful
features. Next, categorical features are converted to numerical and normalized
to enable each feature to contribute equally to anomaly detection. The pre-
processing stage is followed by a decision engine, which may have a training
phase and a validation and test phase. There are many AD methodologies in NAD,
including classification-based methods such as support vector machines and
neural networks, clustering-based methods, deep-learning methods (Sohn, 2021),
knowledge-based methods, combination-based and statistical methods such as
kernel density estimates and particle filters. The authors discuss these
approaches and the available datasets.

More recent work includes a variety of methods ranging from ensemble methods
to fuzzy logic. Zhou et al. (2020) propose an ensemble technique using a
modified adaptive boosting method called M-AdaBoost-A to detect intrusions.
They extend AdaBoost-A, which can handle class imbalanced data to multi-class
classification, and evaluate two variants of the algorithm. Imrana et al.
(2021) propose a deep learning approach – a bi-directional LSTM (long short-
term memory) – for intrusion detection. They evaluate their method using the
NSL-KDD dataset. Hamamoto et al. (2018) propose a method using genetic
algorithms and fuzzy logic for network anomalous event detection. Their
approach consists of two phases: the genetic algorithm is used to create a
digital signature of the network segment and fuzzy logic is used to identify
anomalies. Khan et al. (2021) propose a deep learning method for network
anomaly detection using network spectrogram images generated from Fourier
transforms. They train a deep convolutional neural network to identify the
anomalies. Liu et al. (2021) introduce a NADS using computer log sequences.
They discuss the challenges of vectorizing unstructured log messages while
preserving semantics in an efficient way.

###  2.3 Extreme Value Theory

Extreme Value Theory (EVT) is a branch of statistics used to model rare,
extremal events such as 100-year floods and catastrophic financial losses
(R.D. Reiss and M.Thomas, 2001). Intuitively, EVT focuses on maxima or minima
and represents them using probability distributions. EVT is a powerful, yet
flexible technique to model different types of extremal behavior. From a
statistical point of view, extremes are found in the tail of a distribution.
If the tail exhibits exponential decay, that is, if the tail behavior can be
written as $\lambda e^{-kx}$ for $\lambda,k>0$ where $x$ is the random
variable, then that is good news because many methods can be used to model the
tail behavior and hence we have a handle on the extremes. For example, the
normal distribution has an exponentially decaying tail. The problem arises
when we have heavy or fat tails, that is, when the tail behavior exhibits a
power law relationship or decays polynomially. When this is the case, the tail
behavior can be expressed as $\lambda x^{-k}$ with $\lambda,k>0$. Power law
decay is more challenging than exponential decay because it is difficult to
control or predict the extremes, meaning there is high probability of getting
extreme values. In particular, network traffic is known to have heavy tails
(Hernández-Campos et al., 2004; Ramaswami et al., 2014). Therefore, it is
important to use methodologies such as EVT capable of handling heavy tailed
distributions.

Recent years have seen increasing interest in EVT based methods for anomaly
detection (Kandanaarachchi and Hyndman, 2021; Talagala et al., 2020). We use
Lookout (Kandanaarachchi and Hyndman, 2021), an EVT based AD method to detect
anomalies in honeypot aided computer networks. Lookout has two main
characteristics that are advantageous in a computer network security context:
1. low false positives and a guarantee on false positive probability and  2.
flexibility to model data without an assumption on the underlying probability
distribution.

A low number of false positives is a fundamental characteristic of EVT based
AD methods attributed to its theoretical framework. In addition, Lookout gives
a guarantee or an upper bound on the false positive probability, which can be
set by the user. Furthermore, Lookout is not limited by any distributional
assumptions on the original data. EVT can handle truncated tails,
exponentially decaying tails, polynomially decaying tails or other types of
tail distributions. Depending on the tail of the original distribution, EVT
characterizes extremal distributions into 3 types; Gumbel, Frechet and
Weibull. These 3 distributions fully describe the space of extreme value
distributions, i.e. all extremes fall into one of the above three categories.
A generalized extreme value distribution incorporates these 3 distributions
into 1 distribution using an additional shape parameter $\xi$. Lookout
estimates this shape parameter using the data, thus choosing the appropriate
distribution to model extremes. Therefore, if the data exhibits polynomial
decay written as $\lambda x^{-k}$, then Lookout estimates $k$ from the data,
which is then used to model extremes and identify anomalies.

##  3 Datasets

Figure 4: The monitoring device map for the LAN-Security Monitoring Project.
The nodes are overlaid on a map taken from the Microsoft security intelligence
report (Kelley, 2019) showing average monthly malware encounter rate of
regions.

The LAN-Security Monitoring Project (Ochiai, 2020) is a research collaboration
between 12 ASEAN and SAARC countries led by Japan. The project was deployed in
November 2018, and aims to boost cyber-readiness and cyber-resilience among
partners through research collaboration especially focusing on the security of
local-area networks (LAN). Malware or worms can easily intrude into LANs
through phishing-emails or WiFi, even if the network is protected by a
firewall or operated under a network address translation (NAT) unit. This
project deploys local monitoring devices, honeypots, in LANs because LAN-
internal events, such as direct communications between the devices inside the
LAN are not visible by the global Internet observatories. Monitoring devices
are installed in severe malware-infected countries as shown in Figure 4.

Table 1: Dataset description – features by protocol Protocol |  Feature |  Description
---|---|---
ARP |  timestamp |  The start of the $n$ second time interval for $n\in\\{5,60,600\\}$.
|  node address |  The MAC address of the source node.
|  count |  The number of (broadcast) ARP requests made by the node
|  degree |  The number of IP addresses the node tried to resolve in the time interval of the ARP broadcast request.
TCP |  timestamp |  The start of the 5 second time interval.
|  node address |  The MAC address of the source node.
|  num_ports |  The number of ports specified by the TCP packets from this node within the given time interval.
|  count |  The number of TCP packets the monitoring unit observed from this node.
|  avg_len |  The average length of IP packets that deliver the concerned TCP packets
|  count_fin |  The number of TCP packets with raised fin_flag.
|  count_syn |  The number of TCP packets with raised syn_flag.
|  count_rst |  The number of TCP packets with raised rst_flag.
|  count_psh |  The number of TCP packets with raised psh_flag.
|  count_ack |  The number of TCP packets with raised ack_flag.
|  count_urg |  The number of TCP packets with raised urg_flag.
|  count_ece |  The number of TCP packets with raised ece_flag.
|  count_cwr |  The number of TCP packets with raised cwr_flag.
UDP |  timestamp |  The start of the 5 second time interval.
|  node address |  The MAC address of the node.
|  num_ports |  The number of ports specified by the UDP packets from this node within the given time interval.
|  count |  The number of UDP packets the monitoring unit observed from this node.
|  avg_len |  The average length of IP packets that deliver the concerned UDP packets.

The datasets we obtained pertained to a single LAN with a honeypot for the
time period starting from January 11 2019 until November 15 2020. The honeypot
was quiet and did not make any intentional announcements (e.g., advertisements
or discovery requests) to the network. However, a suspicious node on the same
network segment sometimes directly sends TCP or UDP packets targeting the IP
address of the honeypot. Such a node is considered anomalous as there is no
legitimate reason to send a TCP/UDP packet to the honeypot. This activity is
captured by the software as the packets are directly exchanged with the
honeypot. In addition, all broadcast or multicast Ethernet frames including
ARP requests, DHCP, NBNS, mDNS, LLMNR packets originating from individual
hosts connected to a LAN are also captured by the software. The raw data is
then preprocessed to generate features by protocol, which are given in Table
1.

As mentioned above, the datasets we received from the LAN-Security Monitoring
Project comprised features by protocol for a specific LAN. As listed in Table
1, in addition to TCP/UDP data, ARP data was collected at 5s, 60s and 600s
time intervals. For each time interval, the node address, count and degree are
provided in the dataset.

##  4 Methodology

To consider appropriate methodology, we take into account that the datasets
span 22 months. Using all the data in one batch and detecting anomalous nodes
would amount to detecting anomalies at the end of the 22 month period.
Instead, we use a more continuous approach to detect anomalies using a window-
based model. A window-based approach uses the data in the time window to
detect anomalies. That is, it does not see data outside the window. Both
horizontal and vertical approaches use window-based models for anomaly
detection. We combine the anomalies found in the amalgamation stage.

###  4.1 Sliding and expanding time window models

$t_{e_{i}}$$t_{s_{i}}$$\Delta T$$t_{e_{i+1}}$$t_{s_{i+1}}$$\Delta t$$\Delta
t$Window $i$Window $(i+1)$(a)$t_{e_{i}}$$t_{e_{i+1}}$$\Delta t$(b)Window
$i$Window $(i+1)$ Figure 5: Sliding and expanding windows: (a) The sliding
window model. This is used in the horizontal approach and in the vertical
approach for ARP. (b) The expanding window model. This is only used for TCP
and UDP in the vertical approach.

Figure 5 shows the different window models used for horizontal and vertical
approaches. For the horizontal approach, we use a sliding window model for all
three protocols. We denote the window width by $\Delta T$ and the step size by
$\Delta t$. Suppose the $i^{\text{th}}$ time window starts at $t_{s_{i}}$ and
ends at $t_{e_{i}}$. Then $t_{e_{i}}=t_{s_{i}}+\Delta T$. The
$(i+1)^{\text{st}}$ time window would start at $t_{s_{i+1}}=t_{s_{i}}+\Delta
t$ and ends at $t_{e_{i+1}}=t_{s_{i+1}}+\Delta T=t_{e_{i}}+\Delta t$. This is
illustrated in Figure 5(a). Thus, the horizontal approach considers data in
the time window $\left[t_{s_{i}}\,,t_{e_{i}}\right]$ for each window $i$.

The vertical approach is somewhat different because it considers each protocol
separately to detect anomalies. We use two different window models for the
vertical approach, based on the protocol. For ARP, we use the same sliding
windows as in the horizontal approach. For TCP and UDP we use an expanding
window model shown in Figure 5(b). The honeypot captures all the broadcast ARP
requests, but only captures TCP or UDP packets directed at it. Consequently,
the number of TCP and UDP packets, compared to ARP packets, are low. Thus, a
sliding window model is not suitable for TCP and UDP separately, for the
vertical approach. Thus, for each ARP window ending at time $t_{e_{i}}$, we
use TCP and UDP data for $t\leq t_{e_{i}}$.

While there is no definitive guide to choosing the window size $\Delta T$ or
the step size $\Delta t$, often application specific considerations and trial
and error are used in selecting these parameters. For each window $i$, that
is, at every $t_{e_{i}},$ Honeyboost outputs a list of anomalous nodes
identified from the data in that window. As such, the step size $\Delta t$ is
the frequency of Honeyboost output, which can selected according to
requirement. We have used $\Delta t=60\times 60$ seconds in this experimental
study, i.e., we have configured Honeyboost to output anomalous nodes every
hour.

In contrast, the window size $\Delta T$ determines the amount of data the
algorithm looks at to detect outliers. Smaller window sizes give rise to
higher fluctuations. For example, using a window size of 1 hour results in the
volume of data being high at peak activity hours and much lower at 4 am in the
morning. This is known as seasonality of the data. Often with network traffic
data, there could be multiple seasonal patterns ranging from hourly patterns
to daily patterns. Similar to the hourly fluctuations, there is lower traffic
on some days of the week compared to other days. Therefore, it is important to
choose a window size not affected by such seasonal patterns. In addition, the
higher fluctuations in data volume from smaller window sizes give rise to
higher false positives. Here, to mitigate seasonal fluctuations and to keep
the false positives low, we use $\Delta T=1\,\text{week}=7\times 24\times
60\times 60$ seconds, and configure Honeyboost to predict anomalous nodes
every hour from one week’s worth of data ending at that hour.

###  4.2 Horizontal approach

Figure 6: The input and output data type of each process in the horizontal
approach.

Using the sliding window model, the pre-processed features for all three
protocols, by node, are sorted by time, resulting in, for each time window and
each node, a set of varying dimensional (VD) time series as depicted in Figure
1. Figure 6 illustrates the processes involved in this approach, and the input
and the output data type for each process.

Firstly, the VD time series is reshaped to a multivariate time series using
homogenization (see Table 2). This essential step results in transporting the
research problem to a space where we can use existing time series analysis
methodology. However, finding anomalous time series from a collection of
multivariate time series is a challenging task. Hence, we transform each
multivariate time series to a point in high dimensional space using
metamorphosis (Figure 7), a feature-based time series approach popular in time
series forecasting. This step transforms the problem of finding anomalous
multivariate time series to finding anomalous points in Euclidean space, an
easier problem. However, anomaly detection in high dimensional data is prone
to error due to the curse of dimensionality. Therefore, the dimensions of this
data are reduced using Principal Component Analysis (PCA) and only the first 2
PC scores are kept. Anomalies are then detected in this 2-dimensional PC space
using Lookout and a One Class Support Vector Machine (OCSVM). Lookout is an
Extreme Value Theory based anomaly detection method and achieves lower false
positives compared to the OCSVM. These processes are detailed below.

####  4.2.1 Homogenization: constructing a multivariate time series from a VD
time series

A multivariate time series is constructed using the VD time series for each
node. This transformation, called homogenization, is shown in Table 2 via an
example. The resulting multivariate time series contains 6 numeric features
characterizing important aspects of each protocol along with the node address,
timestamp and protocol name.

As seen in Table 1 the dataset from each protocol contains the node address,
the timestamp and some other numerical features for each observation. These
numerical features are depicted in parenthesis in the example VD time series
given in Table 2. In order to give each protocol the same importance, we
represent the numeric features of each protocol by two key features. The two
numeric features of ARP (see Table 1) namely degree and count are included as
part of the 6 features in the multivariate time series.

TCP, on the other hand, has 11 numeric attributes per node at each timestamp
(see Table 1). We reduce the dimension of this 11-dimensional feature space to
2, by using Principal Component Analysis (PCA). The first 2 principal
component scores PC1 and PC2 for TCP then become part of the 6 features in the
multivariate time series. Similarly UDP has 3 numeric attributes at each
timestamp; PCA is performed for the UDP space and the first 2 PC scores used.

Thus, each node is homogenized to have the following 8 features:  1.
timestamp;  2. protocol name;  3. ARP degree;  4. ARP count;  5. TCP PC1;  6.
TCP PC2;  7. UDP PC1; and  8. UDP PC2.  Thus a VD time series for a given node
with observations at $k$ different time points is transformed into a $k\times
8$ matrix denoting an 8 dimensional time series with $k$ time points. The node
address is a grouping attribute that does not participate in the statistical
analysis. That is, once the anomalous points are identified, the node address
is only used to recognize the responsible node.

Table 2: Homogenization: An example VD time series (left) reshaped to a
multivariate time series (right)

Node Address |  Timestamp |  Protocol | Features
---|---|---|---
N1 | 30 | ARP | $(10,12)$
N1 | 55 | TCP | $(80,2,6,0,2,0,0,0,0,1,1)$
N1 | 85 | UDP | $(3702,2,652)$

$\Longrightarrow$ Node Address Timestamp Protocol ARP Count ARP Degree TCP PC1
TCP PC2 UDP PC 1 UDP PC2 N1 30 ARP 10 12 0 0 0 0 N1 55 TCP 0 0 2.1 1.7 0 0 N1
85 UDP 0 0 0 0 3.6 0.4

####  4.2.2 Metamorphosis: Mapping a time series to a point in high-
dimensional space

TimeTime Series$x_{1}$$x_{2}$$x_{3}$•Metamorphosis Figure 7: A simpler version
of metamorphosis: The figure on the left shows a univariate time series, which
gets mapped to a point in $\mathbb{R}^{3}$ on the right. In reality, we
consider a multivariate time series for each node, and map it to a point in
$\mathbb{R}^{17}$.

For each time window $i$, where $t\in\left(t_{s_{i}},t_{e_{i}}\right)$, and
each node, we consider a multivariate time series. This gives us $m$
multivariate time series if there is traffic from $m$ nodes present in that
time window. We then compute features and transform this time series into a
point in a high-dimensional space. As a time series gets transformed into a
point, we call this process metamorphosis. Figure 7 illustrates this process.
There is a bijective mapping between the nodes and the points in the high-
dimensional space, wherein each node is uniquely identified by a point in the
high dimensional space and each point in the high-dimensional space denotes
the behavior of a specific node in that time window.

The features
For any given node $N_{j}$ the following features are computed from the
multivariate time series associated with $N_{j}$:

  1. 1.

The maximum time difference for the time series $\max t-\min t$ for
$t\in\left(t_{s_{i}},t_{e_{i}}\right)$ for node $N_{j}$

  2. 2.

The number of protocols used by $N_{j}$.

  3. 3.

The number of TCP calls made by $N_{j}$.

  4. 4.

The number of UDP calls made by $N_{j}$.

  5. 5.

The total length of the line segments in $\mathbb{R}^{6}$ using the 6 protocol
features, ARP count, ARP degree, TCP PC1, TCP PC2, UDP PC1 and UDP PC2:
Suppose $\\{\bm{x}_{t_{k}}\\}_{k=1}^{\ell}$ denotes the time series for
$N_{j}$ in $\mathbb{R}^{6}$. The standard Euclidean norm gives the length of
the line segments, viz.,
$\sum_{k=1}^{\ell-1}\lVert\bm{x}_{t_{k+1}}-\bm{x}_{t_{k}}\rVert$. In the
example given in Table 2 the total length of the line segment in $R^{6}$ would
be
$\lVert(10,12,0,0,0,0)-(0,0,2.1,1.7,0,0)\rVert+\lVert(0,0,2.1,1.7,0,0)-(0,0,0,0,3.6,0.4)\rVert$.

  6. 6.

The total length of the line segments in the ARP space spanned by ARP count
and ARP degree: While similar to the previous feature this only takes the
length of the line segments using the ARP count and ARP degree. Figure 8(a)
illustrates this ARP space for a hypothetical node. At times $t_{1}$, $t_{2}$,
$t_{3}$, $t_{4}$, $t_{5}$, $t_{6}$ and $t_{8}$ this node has made ARP calls.
The total length of the line segments is given by $\sum_{t,t^{\prime}\in
S}\lVert\bm{x}_{t^{\prime}}-\bm{x}_{t}\rVert$, where
$S=\\{t_{1},t_{2},t_{3},t_{4},t_{5},t_{6},t_{8}\\}$ and $t$ and $t^{\prime}$
denote successive time points in $S$.

  7. 7.

Next we fit a line to the points in the ARP space using linear regression. We
denote the points in the ARP space by $\\{\bm{x}_{t_{k}}\\}_{k=1}^{\ell}$
where $\bm{x}_{t_{k}}=\left(x_{kc},x_{kd}\right)$ where $x_{kc}$ denotes the
count and $x_{kd}$ denotes the degree at time $t_{k}$. Then the line of best
fit is given by the equation

$\hat{x}_{kd}=m{x}_{kc}+c\,,$ (1)

where $m$ and $c$ denotes the slope and intercept, which are given by

$\displaystyle m$
$\displaystyle=\frac{\ell\sum_{k}{x}_{kc}{x}_{kd}-\sum_{k}{x}_{kc}\sum_{k}{x}_{kd}}{\ell\sum_{k}{x}_{kc}^{2}-\left(\sum_{k}{x}_{kc}\right)^{2}}\,,$
(2) $\displaystyle\text{and}\qquad c$
$\displaystyle=\frac{1}{\ell}\left(\sum_{k}{x}_{kd}-m\sum_{k}{x}_{kc}\right)\,.$
(3)

The quantity $\hat{x}_{kd}$ is the predicted ARP degree by this linear model.
The sum of squares errors is given by

$\text{SSE}=\sum_{k}\left(\hat{x}_{kd}-{x}_{kd}\right)^{2}\,,$ (4)

that is the squared difference between the actual and the predicted degree
values summed over the number of ARP calls. A low value of SSE indicates that
the line is a better fit for the data compared to a high SSE value. These
three features, slope, intercept and SSE, are added to the feature pool. An
example line of best fit is shown in Figure 8(c).

  8. 8.

The equivalent of features 6 and 7 are computed for TCP. This gives the total
length of the line segments in the TCP PC1, PC2 space along with the slope,
intercept and sum of errors squares of the line of best fit (see Figure8(b)).

  9. 9.

Similarly, the length of the line segments in the UDP PC1, PC2 space and the
slope, intercept and SSE of the line of best fit are computed.

We now have 17 features describing the multivariate time series for each node,
thus transforming each time series to a point in $\mathbb{R}^{17}$. Therefore,
a time window containing data from $m$ distinct nodes gives rise to $m$ time
series, which get transformed to $m$ points in $\mathbb{R}^{17}$, with each
point denoting the behavior of a node in that time window.

ARP countARP degree$t_{1}$$t_{2}$$t_{3}$$t_{4}$$t_{5}$$t_{6}$$t_{8}$(a)TCP
PC1TCP PC2$t_{2}$$t_{4}$$t_{5}$$t_{9}$(b)ARP countARP
degree$t_{1}$$t_{2}$$t_{3}$$t_{4}$$t_{5}$$t_{6}$$t_{8}$(c) Figure 8: (a) The
ARP degree and count for a hypothetical node at different time points, with
successive time points connected by line segments. The total length of these
line segments is a feature. (b) The TCP PC1 and PC2 coordinates for the same
node, some of which occur at different time points; the total length of these
line segments is calculated. (c) The line of best fit for points in the ARP
space. The slope, intercept and the sum of squares errors are calculated using
this line of best fit.

####  4.2.3 Dimension reduction and anomaly detection: Identifying anomalies
after dimension reduction

In higher dimensions it is hard to differentiate anomalies from non-anomalies
because points are far away from each other. Therefore, we reduce dimensions
using PCA and use the first 2 dimensions in the PC space for anomaly
detection. We use the algorithm Lookout (Kandanaarachchi and Hyndman, 2021)
and an OCSVM to identify anomalies in this two dimensional space. Lookout uses
Extreme Value Theory (EVT) (Coles, 2001) and leave-one-out kernel density
estimates to identify anomalies. We briefly describe the algorithm Lookout
next.

  1. 1.

To compute kernel density estimates, a bandwidth parameter is required.
However, the bandwidth suited for representing the data in general is not
suitable for anomaly detection. Using persistent homology (Ghrist, 2008), a
methodology in topological data analysis, Lookout chooses a bandwidth
appropriate for anomaly detection.

  2. 2.

Using this bandwidth, kernel density estimates (kde) and leave-one-out kernel
density estimates are computed for the data.

  3. 3.

As anomalies are rare, they lie in low density regions in a feature space. The
bandwidth selected in step 2 results in anomalies having low kde and leave-
one-out kde values. However, we want to identify the anomalies, not just give
an anomaly score for all the data points. A threshold is needed so that
anomalies with kde values lower than this threshold are declared anomalous. We
set a threshold by computing the probabilities of the points using EVT.
Specifically, by using Generalized Pareto Distributions.

  4. 4.

If the probability of a point is below a predefined threshold $\alpha$, it is
declared anomalous.

For this work we set the threshold as $\alpha=0.1$, i.e., if the probability
$p$ of a point being present using the Generalized Pareto Distribution is less
than 0.1, we identify it as an anomaly. Using this probability $p$ the anomaly
score is given by

$s=(0.1-p)\times 100\,,$ (5)

resulting in the anomaly scores lying between $0$ and $10$ with data points
with zero probability having an anomaly score of 10. See Kandanaarachchi and
Hyndman (2021) for more details on Lookout.

Detecting a stream of false anomalies decreases confidence in the NAD system
making a technique with low false positives attractive in this domain. EVT
related methods only identify extremes as anomalies. As such, we expect
Lookout to have a low false positive rate. One-class SVMs (OCSVM) are often
used for anomaly detection in an unsupervised setting. Given the challenges
posed by high false positives in NAD, we compare the performance of Lookout
with an OCSVM.

As the sliding time windows overlap, some nodes are identified as anomalies in
multiple time windows, while others appear as anomalous only in one time
window. If a node is identified as anomalous in multiple time windows, it is a
persistent anomaly, and provides grounds for investigation.

###  4.3 Vertical Approach

The horizontal approach investigates the activity of each node as a time
series and finds anomalous time series. In contrast, the vertical approach
focuses on each protocol to find anomalous activity. All observations in a
single protocol have the same number of attributes, and thus the same
dimension, making the vertical approach much simpler as it does not face the
challenge of assembling varying dimensional data into a single construct. The
vertical approach comprises the following steps:

  1. 1.

The numerical feature space for each protocol has different dimensions, with
ARP being 2-dimensional feature space, TCP 11-dimensional and UDP
3-dimensional (see Table 1). Identifying anomalies in high dimensions is
difficult, and we consider a 2-dimensional space for each protocol. For ARP we
take the count and the degree. For TCP and UDP we perform PCA and take the
first two PC scores.

  2. 2.

Lookout and an OCSVM are used to find anomalies in each protocol’s 2D space.

  3. 3.

The anomalies are combined by node and time window to gain better insights.

For the vertical approach, each point in a protocol 2D space represents a
single communication from a node using that protocol. Nodes communicating
multiple times in a time window are represented by multiple points in the
vertical approach. Thus, if a node $N$ communicates using ARP, TCP and UDP
$m_{1}$, $m_{2}$ and $m_{3}$ times respectively, it will result in $m_{1}$
points in the ARP space, $m_{2}$ points in the TCP space and $m_{3}$ points in
the UDP space.

Finally, the anomalies identified from the horizontal and vertical approaches
for each time window are amalgamated to get a better understanding of the
behavior of the nodes. A node identified as anomalous by both horizontal and
vertical approaches gains higher priority for further inspection than a node
identified by a single approach.

##  5 Results

This section details the results found from using Honeyboost to identify
anomalous nodes. We start with comparing the two AD methods used in Honeyboost
with regards to early detection of anomalous nodes.

###  5.1 Early detection

The two AD methods in Honeyboost – Lookout and OCSVM – perform similarly and
identify most nodes as anomalous before they access the honeypot. Table 3
shows the anomalous nodes, the earliest time they access the honeypot and the
results of the 2 AD methods. The Table gives the earliest time each AD method
identifies a node as anomalous. If this detection happens before the node
accesses the honeypot, it is labeled as early detection, denoted by Early
under the heading Status. The table also indicates the time each node is
identified as anomalous under the heading Time Difference with positive values
indicating early detection. As the window step size is 3600 seconds, any time
difference in the interval $[-3600,0)$ indicates the anomaly was identified in
the same time window.

Of the 15 nodes that access the honeypot, Lookout identifies 13 nodes before
they access the honeypot and 2 nodes (N220 and N225) in the same time window
that they access the honeypot. The OCSVM identifies all nodes before they
access the honeypot.

Table 3: Early detection of honeypot anomalies - amalgamated results using the horizontal and vertical approaches (Early = Early Detection, SW = Same Window) Node |  Earliest Honeypot Time | Lookout Results |  | OCSVM Results
---|---|---|---|---
Earliest Anomaly Detection Time |  Status |  Time Difference (Positive = Early) |  |  Earliest Anomaly Detection Time |  Status |  Time Difference (Positive = Early)
N021 |  1598842055 |  1550034180 |  Early |  48807875 |  |  1547269200 |  Early |  51572855
N039 |  1553585825 |  1553585820 |  Early |  5 |  |  1547449140 |  Early |  6136685
N046 |  1553751195 |  1549530120 |  Early |  4221075 |  |  1547521980 |  Early |  6229215
N132 |  1566462345 |  1561276560 |  Early |  5185785 |  |  1554347700 |  Early |  12114645
N135 |  1554351595 |  1554351540 |  Early |  55 |  |  1554351540 |  Early |  55
N153 |  1563528080 |  1555997940 |  Early |  7530140 |  |  1554803160 |  Early |  8724920
N155 |  1565313000 |  1557801000 |  Early |  7512000 |  |  1554952440 |  Early |  10360560
N158 |  1555312055 |  1555311960 |  Early |  95 |  |  1555311900 |  Early |  155
N159 |  1555313055 |  1555312740 |  Early |  315 |  |  1555312740 |  Early |  315
N163 |  1567236235 |  1558260780 |  Early |  8975455 |  |  1557228000 |  Early |  10008235
N171 |  1565059350 |  1563260880 |  Early |  1798470 |  |  1558065360 |  Early |  6993990
N219 |  1564535840 |  1564535760 |  Early |  80 |  |  1564535760 |  Early |  80
N220 |  1565758430 |  1565758435 |  SW |  -5 |  |  1564552620 |  Early |  1205810
N225 |  1565758055 |  1565758080 |  SW |  -25 |  |  1565695320 |  Early |  62735
N239 |  1575507740 |  1572234480 |  Early |  3273260 |  |  1568005320 |  Early |  7502420

Given that Honeyboost identifies anomalies before they access the honeypot, it
is important to consider the number of false positives it generates. Note that
if a node is tagged as anomalous the first time it communicates with another
node, Honeyboost would detect all anomalies early. However, such a strategy
would create a barrage of false positives, greatly reducing the confidence in
the system. For example, a home security system that constantly trips the
alarm. Therefore, it is imperative to consider the false positive rate, i.e.,
early detection is valuable only if the false positive rate is low.

###  5.2 False positives

Figure 9: False positives using Lookout and the OCSVM for all time windows.
Figures (a) and (c) give the number of false positives for the vertical and
horizontal approaches. Figures (b) and (d) give the false positive rates for
the two approaches.

We investigate the number of false positives and the false positive rate,
defined as the ratio of the number of false positives to the total number of
negatives for both the horizontal and the vertical approaches. Figure 9 shows
the false positives and the false positive rates for both Lookout and OCSVM
for both approaches. Figures 9(a) and (c) show boxplots of the number of false
positives for each time window. The false positive rate is given in Figures
9(b) and (d).

Lookout achieves a low false positive rate compared to the OCSVM for both
approaches. In the vertical approach, the OCSVM identifies many nodes as
anomalies in each window with a median of 20 false positives. In some windows
the OCSVM outputs 80 false positives. This large number of false positives
causes the false positive rate to have a median of 37% and an upper quartile
of 87%. The highest false positive rate for the OCSVM for the vertical
approach is close to 1.

It is interesting to note that the vertical approach produces more false
positives for the OCSVM compared to the horizontal approach. This is due to
the number of data points in a time window for each approach. Suppose a given
time window has $N$ ARP calls belonging to $n$ nodes with $N>>n$. Generally
$N$ is orders of magnitude higher than $n$. The horizontal approach constructs
a data point for each node in each window giving rise to $n$ data points. In
contrast, the vertical approach has $N$ data points. Thus, it is clear the
OCSVM identifies more nodes as anomalous when there are a large number of data
points.

Given that Lookout identifies anomalies early and still has a low false
positive rate, we focus on the results produced by Lookout for the remainder
of the paper.

###  5.3 Amalgamating horizontal and vertical anomalies

For each window, the anomalies detected by the horizontal and vertical
approaches are amalgamated. Table 4 shows the anomalies identified and
amalgamated for the vertical and horizontal approaches for each of 4 windows,
using Lookout. In window 2270, 2 anomalies were identified: N046 is identified
by both the horizontal and vertical approaches, while N157 is identified only
by the vertical approach. Window 5080 also produces 2 anomalies: N220 and
N225, both of which are identified only by the horizontal approach. Windows
7700 and 14240 identify nodes N239 and N021 respectively.

Table 4: Amalgamation of horizontal and vertical anomalies for 4 windows. Window |  Node |  Horizontal Anomaly |  Vertical Anomaly |  Horizontal Anomaly Score |  Vertical Anomaly Score |  Anomaly History (previous windows)
---|---|---|---|---|---|---
2270 |  N046 |  $\qquad\checkmark$ |  $\qquad\checkmark$ |  10 |  19.9 |  2269, 2268, 2267, …
|  N157 |  – |  $\qquad\checkmark$ |  – |  11.8 |  2269, 2268, 2267, …
5080 |  N220 |  $\qquad\checkmark$ |  – |  10 |  – |  5079, 5078, 5077, …
|  N225 |  $\qquad\checkmark$ |  – |  10 |  – |  5079, 5078, 5077, …
7700 |  N239 |  – |  $\qquad\checkmark$ |  – |  10 |  7610, 7586, 7585, …
14240 |  N021 |  $\qquad\checkmark$ |  $\qquad\checkmark$ |  9.97 |  9.93 |  14239, 14238, 14237, …

From Table 4, given Lookout’s low false positive rates, it is clear that every
anomalous node identified, even by a single approach, horizontal or vertical,
should be further investigated. We do this by comparing the results of Table 4
with that of Table 3. Of the anomalies identified in window 2270, for example,
N046 is a real anomalies – we regard it as suspicious because it accesses the
honeypot. N157 is a false positive as it does not access the honeypot.
Similarly, in window 5080, both N220 and N225 are real anomalies, as is the
case with N239 and N021.

##  6 Insights and Discussion

Lookout is the preferred anomaly detection method employed in Honeyboost. As
shown in Figure 2, Honeyboost encompasses the complete framework, starting
from the pre-processing and ending with amalgamating the anomalies that are
found by vertical and horizontal approaches.

###  6.1 Tracking anomalies as they develop over time

The amalgamation of the horizontal and vertical approaches lets us observe the
anomalies as they develop in different spaces. For example, a node may be
identified as anomalous by the horizontal approach and later identified as
anomalous by both approaches. The vertical approach identifies anomalies in
each of the ARP, TCP and UDP spaces separately and then combines them. In
contrast, the horizontal approach identifies anomalies in the combined feature
space.

Figure 10: Anomalies detected by the vertical and horizontal approaches in
windows 1921, 1922, 2089, 2090, 2091, 2109, 2110, 4149, 4150, 4371, 4372,
4651, 4652, 4653, 5186 and 5187. As the vertical approach consists of each of
the ARP, UDP and TCP spaces, anomalies identified separately in these spaces
are included. The vertical axis shows the anomalous nodes and the horizontal
axis gives the windows and anomaly types. For example 1921_H denotes
horizontal anomalies identified in window 1921 and 1921_V_A denotes the
vertical anomalies identified in the ARP space in window 1921. The colors
indicate the logarithm of anomaly scores with darker shades denoting larger
scores.

Figure 10 shows the windows 1921, 1922, 2089, 2090, 2091, 2109, 2110, 4149,
4150, 4371, 4372, 4651, 4652, 4653, 5186 and 5187 and all anomalies identified
in these windows by Lookout. The $Y$ axis shows all nodes labeled as anomalous
in these windows. The $X$ axis gives the different anomaly types sorted by
windows. The anomalies identified in the horizontal and vertical approaches
are differentiated with an _H and _V label, respectively. The vertical
approach anomalies are further classified as ARP, TCP and UDP space anomalies.
Thus, if an anomaly is identified by the vertical approach in the ARP space it
is tagged with _V_A. Similarly, vertical TCP and UDP anomalies are tagged with
_V_T and _V_U, respectively. The $X$ axis labels start with the window number
followed by the anomaly type. Thus, 1921_H denotes horizontal anomalies in
window 1921. Similarly 1921_V_A, 1921_V_T and 1921_V_U denotes vertical ARP,
TCP and UDP anomalies in window 1921. The color of the cells denote the
logarithm of the anomaly scores with darker shades denoting larger scores. The
windows are separated by dashed lines.

We look at an example of a developing anomaly. In window 1921, N135 is the
only node identified as anomalous, and is identified as such by the horizontal
as well as the vertical approach, though only in the ARP space in the latter
case. It is then identified in the vertical UDP space, in window 1922, with a
relatively high anomaly score as seen from the darker color. It is further
identified in window 2089, in the horizontal, as well as vertical ARP and UDP
spaces. Again, in later time windows (4150, 4371), it is identified as
anomalous in the horizontal, and vertical ARP and UDP spaces.

Nodes N158 and N159 are two further examples. They are identified as anomalous
in window 2090 by the horizontal approach, as well as in the vertical UDP
space. These two nodes continue to increase in anomalousness in subsequent
windows as seen by the darker shades of red. In window 2109, both nodes are
identified by the vertical approach in the ARP, UDP and TCP spaces, with high
anomaly scores.

Other examples of developing anomalies in these set of windows are N218 and
N219. Thus, the amalgamation of the horizontal and vertical approaches enables
us to see the anomalies develop, assisting in the decision making process.
Furthermore, Figure 10 shows all anomalies identified by the two approaches in
this set of windows. The maximum number of anomalies identified by Lookout in
this set of windows is 4, occurring in windows 2090 and 4149. In comparison,
the OCSVM identifies an average of 32.7 anomalies per window for this set of
windows, with a minimum of 3 and a maximum of 58 nodes. Having a smaller
number of anomalies increases clarity and helps focus on the important
aspects. This is another advantage of identifying anomalies sparingly.

###  6.2 Tracking a node’s anomalous nature

Figure 11:  The horizontal and vertical anomaly scores for nodes N046, N132,
N153 and N239 over all time windows. The $X$ axis of each graph displays the
range of time windows for which these nodes are identified as anomalous
starting from the window they were first found as anomalous, to the last
window. For example, N132 does not get identified before window 3791. As such,
windows before 3791 are not included in the graph for N132.  Figure 12:
Selected windows for nodes N046, N132, N153 and N239. The anomaly scores for
the horizontal approach, and the vertical approach in the ARP, UDP and TCP
spaces, are shown for each node with dashed lines showing window boundaries.
We see the behavior of the nodes over time.

Next, we focus on individual nodes instead of looking at all anomalies
identified in a given window. Figures 11 and 12 show the behavior of nodes
N046, N132, N153 and N239 over a range of windows along with their anomaly
scores. From Figure 12 we see that these nodes are identified as anomalous
either by a single approach or by both approaches in sets of consecutive
windows. For example, N046 is identified as anomalous in windows 628 to 650,
1656 to 1822 and in windows 2090 to 2277. We see a similar behavior by each of
the other nodes.

This observation is further validated in Figure 12, in which we explore the
ARP, UDP and TCP anomaly spaces for the vertical approach in addition to the
horizontal approach. We only show a subset of windows in Figure 12 due to
space constraints. Node N046 is first identified by the vertical approach in
the ARP space in window 628. Later, in window 1656 it is identified as
anomalous by the horizontal approach as well. This behavior continues for a
while and in window 2118 it is identified by the horizontal approach and the
vertical approach in the ARP and TCP spaces. It continues to be identified in
multiple vertical approach spaces until window 2277.

We see a similar pattern for node N132. Initially it is identified by the
vertical approach in the ARP space in window 3791. In window 5187 it gets
identified by the horizontal approach, and the vertical approach in the ARP,
UDP and the TCP spaces. This continues for many windows and then it slowly
dies down. Nodes N153 and N239 also display a similar pattern where they are
identified by a single approach and then by both approaches in multiple spaces
with higher anomaly scores.

###  6.3 Insights into anomalous behavior

We now combine the results found thus far to gain insights into anomalous
behavior. Table 5 lists the total anomaly score, the number of windows
reporting the node as anomalous, and the average anomaly score per window, of
the nodes accessing the honeypot. The total anomaly score for each node is the
sum of the horizontal and vertical anomaly scores for the full time period.
Figure 13 shows the vertical and horizontal anomaly scores for the top 6
anomalous nodes from this table. We investigate these anomalies to gain better
understanding.

Table 5: Nodes accessing the honeypot sorted by the total anomaly score. The number of windows reporting the node as anomalous as well as the average score per window are also given.  Node |  Horizontal Anomaly Score Total |  Vertical Anomaly Score Total |  Total Anomaly Score |  Number of Anomalous Windows |  Average Score Per Window
---|---|---|---|---|---
N135 |  6998 |  62473 |  69471 |  1351 |  51.4
N159 |  1487 |  30180 |  31666 |  169 |  187.0
N153 |  959 |  23902 |  24862 |  848 |  29.3
N158 |  1517 |  22235 |  23752 |  172 |  138.0
N132 |  4119 |  9305 |  13424 |  551 |  24.4
N219 |  1793 |  7072 |  8865 |  194 |  45.7
N021 |  2615 |  5657 |  8272 |  766 |  10.8
N155 |  1420 |  6002 |  7421 |  427 |  17.4
N046 |  1965 |  3191 |  5157 |  360 |  14.3
N039 |  2092 |  2297 |  4389 |  210 |  20.9
N163 |  1060 |  2670 |  3730 |  152 |  24.5
N239 |  1333 |  1899 |  3232 |  334 |  9.68
N225 |  1070 |  1291 |  2361 |  247 |  9.56
N220 |  1240 |  0 |  1240 |  124 |  10.0
N171 |  940 |  18 |  958 |  98 |  9.78
Figure 13: The horizontal and vertical anomaly scores for the top 6 anomalous
nodes accessing the honeypot, as per Table 5.

  1. 1.

N135: This node suddenly makes a large number of ARP broadcasts (count 757,
degree 257) and accesses the honeypot using UDP and TCP. The total length of
the UDP packets per 5s time interval are unusually high for N135. The median
value of the total length of the UDP packets, taking into account all nodes
accessing the honeypot, is 28. For N135 the total UDP packet length ranges
from 28 to 1047, with a median of 927. N135 has the highest total anomaly
score because it is identified in 1351 windows. From Figure 13 we see that it
displays a lot of activity for a long period of time.

  2. 2.

N159: This node makes innocuous ARP broadcasts for the first 9 minutes of its
communication. Then, it suddenly makes a large number of ARP broadcasts (count
6760, degree 3384) and starts accessing the honeypot immediately after that,
using 32 UDP ports in a single 5 second interval. After a time lag of 1 hour,
it accesses the honeypot using 999 TCP ports. From Figure 13 we see that the
vertical anomaly scores in certain windows are much higher for N159 compared
to other nodes. Furthermore, it is only found anomalous in 169 windows. Thus,
N159 has the highest average anomaly score per window.

  3. 3.

N153: This node is found anomalous in 848 windows. It has a long span of
activity as seen from Figure 13. In terms of the ARP broadcasts, it has a
higher count to degree ratio compared to most other nodes.

  4. 4.

N158: This node has very large ARP counts for a long period of time, with a
maximum ARP count of 6536 and degree 3262 for a given time interval. After
these ARP broadcasts it accesses the honeypot using 32 UDP ports and 999 TCP
ports.

  5. 5.

N132: Again, we see extremely large ARP counts for multiple time intervals.
The maximum ARP count for N132 is 22558 with degree 4416. It then accesses the
honeypot using TCP and UDP protocols. We find that N132 has set the PSH (push)
and the URG (urgent) flags in some TCP packets that access the honeypot. The
PSH flag informs the receiving node that the data should be pushed to the
application layer immediately, and the URG flag informs the receiver that the
data should be prioritized.

  6. 6.

N219: Similar to N132, this node sets the PSH and URG flags for some TCP
packets and accesses the honeypot using a large number of TCP ports.

Certain patterns emerge from this analysis. Some nodes such as N135 and N153
are identified as anomalous in a large number of windows. These nodes access
the honeypot over a long period of time. In contrast, other nodes such as
N219, access the honeypot over a small time interval and are identified as
anomalous with high anomaly scores.

Furthermore, we see different types of suspicious behavior: unusually high
number of ARP broadcasts by a node before accessing the honeypot followed by
TCP or UDP packets targeted at the honeypot using a large number of ports; and
nodes accessing the honeypot using specific TCP flags that requests urgent
attention and access to higher layers. Overall, it is clear that by combining
the horizontal and vertical approaches, Honeyboost, via Lookout, prioritizes
the nodes that need more inspection.

###  6.4 False Positives vs identifying anomalies

Honeyboost identifies all nodes accessing the honeypot. In addition, it
identifies most nodes before they access the honeypot. Of the identified
anomalies, all do not access the honeypot, i.e. some are false positives.
Table 6 shows the top 3 anomalous nodes, N001, N135, and N157, identified by
Honeyboost and their anomaly scores.

Table 6: Top 3 Honeyboost anomalies sorted by the total anomaly score. The number of anomalous windows and average score per window are also given. Node |  Horizontal Anomaly Score Total |  Vertical Anomaly Score Total |  Total Anomaly Score |  Number of Anomalous Windows |  Average Score Per Window
---|---|---|---|---|---
N157 |  230 |  585540 |  585770 |  86 |  6811
N001 |  72178 |  5744 |  77921 |  7445 |  10.5
N135 |  6998 |  62473 |  69471 |  1351 |  51.4

Of these 3 nodes, we discussed N135 in Section 6.3. The other 2 nodes, N157
and N001 do not access the honeypot. However, it is worth discussing their
behavior.

Figure 14 shows the anomaly scores, ARP count and degree values for N157 over
time. From Figure 14(a), we see that the vertical anomaly scores for N157 are
extremely large. Many anomaly scores are greater than 5000, and some scores
are over 40,000. In fact, from Figure 14(a) it is misleading to think that the
horizontal score is zero for all windows. For some windows the horizontal
score is 10, the maximum score for the horizontal approach, but this is
dwarfed by the high values in the vertical axis. The vertical approach does
not have a maximum score as a node can make multiple calls using ARP, UDP and
TCP protocols and if many of them are identified anomalous the sum of the
anomaly scores is considered for each window.

The reason for high anomaly scores from the vertical approach is the large
number of ARP calls with high count that are not balanced by high degree
values as seen in Figures 14(b) and (c). Starting from timestamp 1555482240,
for every 60s interval N157 makes over 110 ARP broadcasts over 6381 minutes
continuously as can be seen from the thick, black line in Figure 14(b). It is
rather unusual for a node to make such ARP broadcasts continuously for a long
period of time. Especially, as it did not make such ARP broadcasts before that
time. Therefore, this node needs investigation.

Figure 14: Behavior of Node N157: (a) Anomaly scores. (b) The ARP count over
time. (c) The ARP degree over time.

Another node of interest is N001. Figure 15(a) shows a histogram of the number
of windows each node is found anomalous. N001 is found anomalous in 7445
windows and is the most frequently identified anomaly surpassing the other
nodes by a large margin. Figure 15(b) shows the anomaly scores for N001 for
the horizontal and vertical approaches. On investigating the feature space, we
find that this node has a relatively large total length of 73.4 in the ARP
space (Section 4.2.2, feature 6). It turns out this node has made 4635 ARP
broadcasts at different time points in its first anomalous time window, all
having count and degree values of 1, 2, or 3. Over the full time period, it
makes 445,100 ARP broadcasts. Therefore, N001 is unusual because it makes a
small number of ARP broadcasts extremely frequently over a long period of
time. It might be a malfunctioning node needing investigation. We also note
that node N001 would not have been identified by simply tagging the nodes
making a large number of ARP broadcasts.

Figure 15: Behavior of Node N001: (a) Histogram of the nodes identified as
anomalies in all time windows with N001 getting identified 7445 times. N001 is
labeled in this histogram. (b) The horizontal and vertical anomaly scores over
all windows for N001.

##  7 Conclusion

In this paper, we presented Honeyboost, a novel, hybrid framework consisting
of two complementary approaches – horizontal and vertical – to enhance
honeypot aided NAD. Both approaches use data fusion techniques to integrate
attributes from diverse protocols. Our methodology does not suffer from a high
false positive rate as we use an anomaly detection method called Lookout,
which uses extreme value theory to identify anomalies. Furthermore, it
operates totally unsupervised, alleviating the need for costly labeling
procedures. Using our framework, we successfully identified anomalous nodes
before they accessed the honeypot. In addition, we gained useful insights
about the behavior of these nodes. Moreover, we identified some unusual
behavior by nodes that do not access the honeypot. As future work, we plan to
investigate network science and graph theory approaches to further boost early
detection and classification of anomalies.

## References

  * (1)
  * Ahmed et al. (2016) Ahmed, M., Naser Mahmood, A. and Hu, J. (2016), ‘A survey of network anomaly detection techniques’, Journal of Network and Computer Applications 60, 19–31.
  * Almohannadi et al. (2018) Almohannadi, H., Awan, I., Al Hamar, J., Cullen, A., Disso, J. P. and Armitage, L. (2018), Cyber threat intelligence from honeypot data using elasticsearch, in ‘Proceedings - International Conference on Advanced Information Networking and Applications, AINA’, Vol. 2018-May, pp. 900–906.
  * Anirudh et al. (2017) Anirudh, M., Arul Thileeban, S. and Nallathambi, D. J. (2017), Use of honeypots for mitigating DoS attacks targeted on IoT networks, in ‘International Conference on Computer, Communication, and Signal Processing: Special Focus on IoT, ICCCSP 2017’.
  * Baddar et al. (2014) Baddar, S. H. A. H., Merlo, A. and Migliardi, M. (2014), ‘Anomaly detection in computer networks: A state-of-the-art review’, Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications 5(4), 29–64.
  * Barak (2020) Barak, I. (2020), ‘Critical infrastructure under attack: lessons from a honeypot’, Network Security 2020(9), 16–17.
  * Baykara and Das (2018) Baykara, M. and Das, R. (2018), ‘A novel honeypot based security approach for real-time intrusion detection and prevention systems’, Journal of Information Security and Applications 41, 103–116.
  * Campbell et al. (2015) Campbell, R. M., Padayachee, K. and Masombuka, T. (2015), A survey of honeypot research: Trends and opportunities, in ‘2015 10th international conference for internet technology and secured transactions (ICITST)’, IEEE, pp. 208–212.
  * Coles (2001) Coles, S. (2001), An introduction to statistical modeling of extreme values, Vol. 208, Springer.
  * Fan et al. (2019) Fan, W., Du, Z., Smith-Creasey, M. and Fernandez, D. (2019), ‘HoneyDOC: An Efficient Honeypot Architecture Enabling All-Round Design’, IEEE Journal on Selected Areas in Communications 37(3), 683–697.
  * Fernandes et al. (2019) Fernandes, G., Rodrigues, J. J., Carvalho, L. F., Al-Muhtadi, J. F. and Proença, M. L. (2019), ‘A comprehensive survey on network anomaly detection’, Telecommunication Systems 70(3), 447–489.
  * Ghrist (2008) Ghrist, R. (2008), ‘Barcodes: the persistent topology of data’, Bulletin of the American Mathematical Society 45(1), 61–75.
  * Goldstein and Uchida (2016) Goldstein, M. and Uchida, S. (2016), ‘A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data’, PLoS ONE 11(4).
  * Hamamoto et al. (2018) Hamamoto, A. H., Carvalho, L. F., Sampaio, L. D. H., Abrão, T. and Proença, M. L. (2018), ‘Network Anomaly Detection System using Genetic Algorithm and Fuzzy Logic’, Expert Systems with Applications 92, 390–402.
  * Handa et al. (2021) Handa, A., Negi, R. and Shukla, S. K. (2021), Part I Deception Technologies amp; Threat Visibility – Honeypots and Security Operations, pp. 3–3.
  * Hawkins (1980) Hawkins, D. M. (1980), Identification of outliers, Vol. 11, Springer.
  * Hernández-Campos et al. (2004) Hernández-Campos, F., Marron, J. S., Samorodnitsky, G. and Smith, F. D. (2004), ‘Variable heavy tails in Internet traffic’, Performance Evaluation 58(2-3), 261–284.
  * Imrana et al. (2021) Imrana, Y., Xiang, Y., Ali, L. and Abdul-Rauf, Z. (2021), ‘A bidirectional LSTM deep learning approach for intrusion detection’, Expert Systems with Applications 185, 115524.
  * Jasek et al. (2013) Jasek, R., Kolarik, M. and T., V. (2013), APT detection system using honeypots, in ‘13th International Conference on Applied Informatics and Communications (AIC’13)’, pp. 25–29.
  * Kandanaarachchi and Hyndman (2021) Kandanaarachchi, S. and Hyndman, R. J. (2021), Leave-one-out kernel density estimates for outlier detection, Technical report, RMIT University.
http://bit.ly/lookoutliers

  * Kelley (2019) Kelley, D. (2019), Microsoft security intelligence report volume 24, Technical report.
https://www.microsoft.com/sir/

  * Khan et al. (2021) Khan, A. S., Ahmad, Z., Abdullah, J. and Ahmad, F. (2021), ‘A Spectrogram Image-Based Network Anomaly Detection System Using Deep Convolutional Neural Network’, IEEE Access 9, 87079–87093.
  * Kondra et al. (2016) Kondra, J. R., Bharti, S. K., Mishra, S. K. and Babu, K. S. (2016), Honeypot-based intrusion detection system: A performance analysis, in ‘2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom)’, pp. 2347–2351.
  * La et al. (2016) La, Q. D., Quek, T. Q., Lee, J., Jin, S. and Zhu, H. (2016), ‘Deceptive Attack and Defense Game in Honeypot-Enabled Networks for the Internet of Things’, IEEE Internet of Things Journal 3(6), 1025–1035.
  * Liu et al. (2021) Liu, X., Liu, W., Di, X., Li, J., Cai, B., Ren, W. and Yang, H. (2021), ‘LogNADS: Network anomaly detection scheme based on log semantics representation’, Future Generation Computer Systems 124, 390–405.
  * Matin and Rahardjo (2020) Matin, I. M. M. and Rahardjo, B. (2020), The Use of Honeypot in Machine Learning Based on Malware Detection: A Review, in ‘2020 8th International Conference on Cyber and IT Service Management, CITSM 2020’.
  * Maxion (1990) Maxion, R. A. (1990), ‘Anomaly detection for diagnosis’, Digest of Papers - FTCS (Fault-Tolerant Computing Symposium) pp. 20–27.
  * Mokube and Adams (2007) Mokube, I. and Adams, M. (2007), Honeypots: Concepts, approaches, and challenges, in ‘Proceedings of the Annual Southeast Conference’, Vol. 2007, pp. 321–326.
  * Moore and Al-Nemrat (2015) Moore, C. and Al-Nemrat, A. (2015), An analysis of honeypot programs and the attack data collected, in ‘Communications in Computer and Information Science’, Vol. 534, pp. 228–238.
  * Moustafa et al. (2019) Moustafa, N., Hu, J. and Slay, J. (2019), ‘A holistic review of Network Anomaly Detection Systems: A comprehensive survey’, Journal of Network and Computer Applications 128, 33–55.
  * Naveed and Wu (2020) Naveed, K. and Wu, H. (2020), ‘Celosia: An Immune-Inspired Anomaly Detection Framework for IoT Devices’, Proceedings - Conference on Local Computer Networks, LCN 2020-November, 13–23.
  * Ochiai (2020) Ochiai, H. (2020), ‘Lan-security monitoring project’, IEICE Technical Report; IEICE Tech. Rep. 120(19), 27–38.
  * Ramaswami et al. (2014) Ramaswami, V., Jain, K., Jana, R. and Aggarwal, V. (2014), ‘Modeling heavy tails in traffic sources for network performance evaluation’, Advances in Intelligent Systems and Computing 246, 23–44.
  * Razali et al. (2019) Razali, M. F., Muruti, G., Razali, M. N., Jamil, N. and Mansor, F. Z. (2019), IoT honeypot: A review from researcher’s perspective, in ‘2018 IEEE Conference on Application, Information and Network Security, AINS 2018’, pp. 93–98.
  * R.D. Reiss and M.Thomas (2001) R.D. Reiss and M.Thomas (2001), Statistical Analysis of Extreme values with Applications to Insurance, Finance, Hydrology and Other Fields, Vol. 44.
  * Ruff et al. (2020) Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G. and Müller, K. R. (2020), ‘A unifying review of deep and shallow anomaly detection’, arXiv .
  * Sadasivam and Hota (2015) Sadasivam, G. K. and Hota, C. (2015), ‘Scalable honeypot architecture for identifying malicious network activities’, Proceedings - 2015 2nd International Conference on Emerging Information Technology and Engineering Solutions, EITES 2015 pp. 27–31.
  * Seungjin et al. (2020) Seungjin, L., Abdullah, A. and Jhanjhi, N. Z. (2020), ‘A review on honeypot-based botnet detection models for smart factory’, International Journal of Advanced Computer Science and Applications 11(6), 418–435.
  * Shrivastava et al. (2019) Shrivastava, R. K., Bashir, B. and Hota, C. (2019), Attack Detection and Forensics Using Honeypot in IoT Environment, in G. Fahrnberger, S. Gopinathan and L. Parida, eds, ‘Distributed Computing and Internet Technology’, Springer International Publishing, pp. 402–409.
  * Sohn (2021) Sohn, I. (2021), ‘Deep belief network based intrusion detection techniques: A survey’, Expert Systems with Applications 167, 114170.
  * Talagala et al. (2020) Talagala, P. D., Hyndman, R. J., Smith-Miles, K., Kandanaarachchi, S. and Muñoz, M. A. (2020), ‘Anomaly Detection in Streaming Nonstationary Temporal Data’, Journal of Computational and Graphical Statistics 29(1), 13–27.
  * Tiruvakadu and Pallapa (2018) Tiruvakadu, D. S. K. and Pallapa, V. (2018), ‘Confirmation of wormhole attack in MANETs using honeypot’, Computers and Security 76, 32–49.
  * Wang et al. (2019) Wang, H., Bah, M. J. and Hammad, M. (2019), ‘Progress in Outlier Detection Techniques: A Survey’, IEEE Access 7, 107964–108000.
  * Zhan et al. (2013) Zhan, Z., Xu, M. and Xu, S. (2013), ‘Characterizing honeypot-captured cyber attacks: Statistical framework and case study’, IEEE Transactions on Information Forensics and Security 8(11), 1775–1789.
  * Zhang et al. (2021) Zhang, J., Bhuiyan, M. Z. A., Yang, X., Wang, T., Xu, X., Hayajneh, T. and Khan, F. (2021), ‘AntiConcealer: Reliable Detection of Adversary Concealed Behaviors in EdgeAI Assisted IoT’, IEEE Internet of Things Journal pp. 1–1.
  * Zhou et al. (2020) Zhou, Y., Mazzuchi, T. A. and Sarkani, S. (2020), ‘M-AdaBoost-A based ensemble system for network intrusion detection’, Expert Systems with Applications 162, 113864.