# E-GraphSAGE: A Graph Neural Network based
Intrusion Detection System

Wai Weng Lo, Siamak Layeghy, Mohanad Sarhan, Marcus Gallagher, , and
Marius Portmann “This work has been submitted to the IEEE for possible
publication. Copyright may be transferred without notice, after which this
version may no longer be accessible.” Corresponding author: Wai Weng Lo
(email<EMAIL_ADDRESS>
###### Abstract

This paper presents a new network intrusion detection system (NIDS) based on
Graph Neural Networks (GNNs). GNNs are a relatively new sub-field of deep
neural networks, which have the unique ability to leverage the inherent
structure of graph-based data. Training and evaluation data for NIDSs are
typically represented as flow records, which can naturally be represented in a
graph format. This establishes the potential and motivation for exploring GNNs
for the purpose of network intrusion detection, which is the focus of this
paper. E-GraphSAGE, our proposed new approach is based on the established
GraphSAGE model, but provides the necessary modifications in order to support
edge features for edge classification, and hence the classification of network
flows into benign and attack classes. An extensive experimental evaluation
based on six recent NIDS benchmark datasets shows the excellent performance of
our E-GraphSAGE based NIDS in comparison with the state-of-the-art.

###### Index Terms:

Graph Neural Networks, Network Intrusion Detection System, Internet of Things

##  I Introduction

Network attacks have significantly increased over the last few years, both in
terms of frequency and level of sophistication. Network intrusion detection
systems (NIDSs), which are placed at strategic points within the network to
monitor traffic, are an important tool for the detection and mitigation of
network-based cyber attacks. There are two main types of NIDS, signature based
and anomaly detection based systems. A signature based NIDS relies on a pre-
installed set of attack signatures, which are compared and pattern-matched
with the monitored network traffic in order to detect attacks. As a result,
this type of NIDS can effectively detect known attacks, with a relatively low
false alarm rate, but it is significantly less effective in detecting new
attacks or variants of existing attacks. In contrast, anomaly detection based
systems have the potential to detect intrusions by detecting deviations from
the regular traffic patterns in the network. Over the last few years, there
has been great progress in the application of new Machine Learning (ML) models
and techniques, in particular deep learning based approaches, for the
development of new NIDS solutions. Several recently proposed ML-based NIDSs
have achieved a high performance for the detection and identification of
network attacks [1, 2, 3].

In this paper, we are exploring the use of Graph Neural Networks (GNNs), a
relatively new sub-field of deep neural networks, for network intrusion
detection. GNNs are tailored to applications with graph-structured data, such
as social sciences, chemistry, and telecommunications, and are able to
leverage the inherent structure of the graph data by building relational
inductive biases into the deep learning architecture. This provides the
ability to learn, reason and generalise from the graph data, inspired by the
concept of message propagation [4].

Network intrusion detection is typically performed on flow based network data
such as NetFlow, where flows are identified by the communication endpoints (IP
address, L4 port numbers, L4 protocol), and annotated via a set of flow fields
which provide details on the flows, such as the number of packets, number of
bytes, flow duration, etc. This flow data can naturally be represented in a
graph format, where the flow endpoints are mapped to graph nodes, and network
traffic flows are mapped to the graph edges.

In this paper, we propose a novel GNN-based approach for network intrusion
detection. Our proposed method is based on the GraphSAGE algorithm [5], that
can be used to generate low-dimensional vector representations (embeddings) of
graph nodes, which in turn can be used for tasks such as node classification.
However, since the original version of GraphSAGE does not support edge
features, it cannot be directly used for flow classification in network
intrusion detection. We propose E-GraphSAGE, a variant of GraphSAGE which
allows collecting information on graph edges, supports edge features for edge
classification, and hence the detection of malicious network flows.

We demonstrate how the E-GraphSAGE algorithm can be utilised to build a
reliable NIDS, and we provide an extensive experimental evaluation of our
proposed system on six recent NIDS benchmark datasets. Our results show that
the E-GraphSAGE-based NIDS achieves excellent classification accuracy, and
outperforms or equals the best reported results for each of the six considered
NIDS datasets, for both binary and multiclass classification, with only two
minor exceptions. To the best of our knowledge, our work is the first
successful, practical and extensively evaluated approach of applying Graph
Neural Networks for the problem of network intrusion detection.

The rest of the paper is organized as follows. Section II discusses related
works, and Section III provides the relevant background on GNNs and GraphSAGE.
Our proposed E-GraphSAGE algorithm and corresponding NIDS are introduced in
Section IV. Experimental evaluation results are presented in Section VI, and
Section VII concludes the paper.

##  II Related Work

Our discussion of related works is organised in two parts. First, we discuss
general ML-based network intrusion detection methods which have been evaluated
on recent NIDS datasets, created in recent years. Secondly, we specifically
discuss approaches that use Graph Neural Network (GNN) approaches for network
intrusion and anomaly detection methods. Since this is a very new area of
research, none of these works have been experimentally evaluated on any of the
six NIDS benchmark datasets considered in this paper. Consequently, we are
limited to a qualitative discussion and comparison of these works with our
proposal.

###  II-A ML-based NIDSs

Mohamed et al. [3] proposed a hybrid deep learning ML-based NIDS using a
combination of Convolutional Neural Network (CNN) and Long Short-Term Memory
(LSTM) architectures. The CNN layers are used to extract important features
from network data due to its speed advantage caused by the weight sharing
property. The LSTM retains the long-term temporal relationship between the
data features. The hyper-parameters of the system were optimised using a trial
and error process. The overall classification accuracy achieved on the UNSW-
NB15 dataset is 97.1% for binary classification, and 98.43% for the multi-
class case.

Haitao et al.[6] designed a multimodal sequential NIDS with a hierarchical
progressive network to capture different levels of network data features. The
approach is based on a combination of deep autoencoder and LSTM architectures,
in order to integrate the structural information within the temporal
information shared between similar network connections. The system is
evaluated on 3 datasets including the UNSW-NB15 dataset, where it achieved an
accuracy of 96.8% and 86.2% in binary and multiclass classification
experiments respectively.

In [7], an attack mitigation framework for IoT networks is proposed using a
hybrid of signature- and anomaly-based detection systems. The signature-based
system uses a database of black-listed sources. The anomaly-based module
adopts an extreme gradient boosting (XGBoost) algorithm. The XGBoost
classifier achieved a binary classification accuracy of 99.99% and a false
alarm rate of 0.05% on the BoT-IoT dataset, and 99.96% multiclass
classification accuracy with 97% F1 score.

In [1], the authors focused on securing Internet of Things (IoT) networks
using a two-level anomaly detection model. The first level categorizes the
network flows as normal or anomalous (attack), while the second level
classifies the attack type. Various ML models were investigated and the
Decision Tree and Random Forest classifiers achieved the highest performance
on the BoT-IoT dataset in level 1 and level 2 classifications, respectively.
The proposed model achieved an accuracy of 99.80% and an F1-Score of 99.80%.

Sarhan et al. [8] converted the UNSW-NB15, BoT-IoT and ToN-IoT datasets from
their proprietary formats and feature set into a common, Netflow-based format,
with eight Netflow features as a common feature set. The corresponding new
Netflow-based variants of the datasets, i.e. NF-UNSW-NB15, NF-BoT-IoT and NF-
ToN-IoT, have been made publicly available. The authors evaluated an Extra
Tree ensemble classifier across these three datasets, and have reported an
F1-Score of 0.85, 0.97 and 1.00 respectively for binary classification. For
the multi-class case, the corresponding F1-Scores are 0.98, 0.77 and 0.60.

###  II-B GNN-based NIDSs

Xiao et al. [9] proposed a graph embedding approach to perform anomaly
detection on network flows. The authors first converted the network flows into
a first-order and second-order graph. The first-order graph learns the latent
features from the perspective of a single host by using its IP address and
port number. The second-order graph aims to learn the latent features from a
global perspective by using source IP addresses, source ports, destination IP
addresses, as well as destination ports. The graph embeddings are then used to
train a Random Forest classifier to detect network attacks. Since the
evaluation only considers a single NIDS dataset, in contrast to the six
datasets considered in this paper, there is limited information on the
generalisability of this approach. A more significant limitation of this
approach is due to the fact that it is using a transductive learning method
[5], which limits its ability to classify samples with previously unseen graph
nodes, e.g. IP addresses and port numbers, which were not seen during the
training phase. This makes the approach unsuitable for many practical NIDS,
since we cannot assume that all possible IP addresses and port numbers that
can occur in a production network would have been available in the training
dataset. In contrast, the approach presented in this paper uses an inductive
learning approach, which does not suffer this limitation.

Zhou et al.[10] proposed using a graph convolutional neural network to perform
botnet detection. They first generated botnet traffic by creating botnet
connections mixed with different real large-scale network traffic flows. Then,
they applied the graph convolutional neural network for network node
classification. The generated graph does not include any flow or node
features, and only considers the topological information of the network
connectivity graph. Furthermore, the approach does not detect individual
attack flows, but is limited to detecting attack nodes, specifically in the
context of botnets. This is in contrast to the method proposed in this paper,
which is able to leverage all the information provided in network flow data,
i.e. graph structure and edge features, and is able to detect individual
attack flows.

In contrast to related works that apply Graph Neural networks (GNN) for NIDS,
as discussed above, the method proposed in this paper is practical due to its
inductive learning approach, and it can deal with new graph nodes that have
not been seen during training. Furthermore, our approach is able to leverage
edge features that can be extracted from network flow data, and is able to
detect individual attack flows via its edge embedding approach. Network
intrusion detecting systems based on traditional (non graph-based) machine
learning algorithms currently do not fully leverage the structural and
topological information that is inherent in network flow data, which is one of
the key strengths of GNNs. A key feature of the method proposed in this paper
is its ability to leverage both topological information, as well as edge
features in network flow data. Furthermore, related works that use an ML-
approach for NIDS typically use one, two or a maximum of three NIDS benchmark
datasets to evaluate the proposed system. For the evaluation of our GNN-based
approach proposed in this paper, we use six different datasets, which provides
a higher degree of confidence in the method’s robustness and the ability to
generalise to different types of network scenarios.

##  III Background

###  III-A Graph Neural Networks (GNN)

Graph Neural Networks (GNN) and graph representation learning is one of the
most recent and fastest growing sub-areas in machine learning. Its power and
potential lies in its ability to leverage the inherent graph structure of a
lot of data encountered in real world application domains, such as social
media networks, biology, telecommunications, etc. The graph format captures
the structural information by modelling a set of objects and their
relationships. The objects are represented by graph nodes and their
relationships by graph edges. In the case of computer networks, individual
hosts (IP addresses) are modelled as graph nodes, and the communication
between the hosts, i.e. the network flows, are modelled as graph edges.

The key motivation of using a GNN for network intrusion detection systems
(NIDSs) is the ability to easily and directly exploit the rich structural
information in the network flow data, which can be directly encoded in a graph
format. While some traditional ML-based approaches attempt to work with graph
data, this is typically quite cumbersome and relies on hand-engineered
features.

Convolutional Neural Networks (CNNs) have been very successfully applied to
data with a rigid and regular structure, such as images. The Graph Neural
Networks (GNNs) can be thought as a generalisation of Convolutional Neural
Networks to non-Euclidean data structures [11].

A common task performed by GNNs is to generating node embeddings, which aims
to encode nodes as low-dimensional vectors, while maintaining their key
relationships and graph position in the original format. A pair of node
embeddings can be concatenated together to form the edge embeddings to
represent the edges. Node or edge embedding is typically a key precursor to
’downstream tasks such as node and edge classification or link prediction.
GNNs have recently received a lot of attention due to their convincing
performance and high interpretability of the results through the visualisation
of the graph embeddings [12].

###  III-B GraphSAGE

In order to generalise capabilities of Convolutional Neural Networks (CNNs) to
data with a non-Euclidean (non-grid-like) structure, Graph Neural Networks
(GNNs) use the concept of message passing or propagation. For this, features
(or attributes) of the neighbours of a graph node are typically aggregated and
passed as a message to that node. This process is repeated over a number of
iterations, to propagate the information from nodes across the network. The
final result, i.e. the aggregate information obtained at each node, is
referred to as the node embedding. If information is collected from all
neighbours of each node at each iteration, as proposed in a lot of GNNs, the
approach suffers from scalability limitations, as well as unpredictable memory
and computational resource requirements on large graphs [5].

The Graph SAmple and aggreGatE (GraphSAGE) algorithm [5] was developed by
Hamilton et al. to address these limitations [5]. In GraphSAGE, instead of
passing messages from all neighbours of a node, only a fixed size sub-set is
(uniformly randomly) sampled. This allows limiting the space and time
complexity of the algorithm, irrespective of the graph structure (e.g. node
degree distribution) and batch size.

Figure 1: A given graph (left), and the corresponding GraphSAGE architecture
with depth-2 convolutions (right) and full neighborhood sampling.

Figure 1 (left) shows a simple example of a graph, and Figure 1 (right) shows
the corresponding GraphSAGE message passing mechanism for two convolutional
layers, where information is propagated across the hops in the graph [5]. In
this particular example, we assume full neighborhood sampling, where
information from all neighbour nodes is considered in each iteration.

The GraphSAGE algorithm operates on a graph
$\mathcal{G}(\mathcal{V},\mathcal{E})$, where $\mathcal{V}$ is the set of
nodes and $\mathcal{E}$ is the set of edges. The node features of a node $v$
are represented as the vector $x_{v}$, and the complete set of node feature
vectors as $\\{x_{v},\forall v\in\mathcal{V}\\}$.

A key hyperparameter of the GraphSAGE algorithm is the number of graph
convolutional layers $K$, which specifies the number of hops via which node
information is aggregated at each iteration. Another important aspect of
GraphSAGE is the choice of a differentiable aggregator function
$AGG_{k},\forall k\in\\{1,...,K\\}$, to aggregate information from neighbor
nodes. The algorithm is performed in both a forward and backward propagation
stage, which we now discuss in more detail.

####  III-B1 Forward Propagation - Node Embedding

Similar to the convolution operation in CNNs, information relating to the
local neighborhood of a node is collected and is used to compute the node
embedding. The GraphSAGE algorithm starts by assuming the model has already
been trained and the weight matrices and aggregator function parameters are
fixed. For each node, the algorithm iteratively aggregates information from
the node’s neighbours, the node’s neighbours-neighbours, and so on.

At each iteration, initially the neighborhood of the node is sampled, and the
information from the sampled nodes is aggregated into a single vector. At the
$k$-th layer, the aggregated information $\mathbf{h}_{N(v)}^{k}$ at a node
$v$, based of the sampled neighborhood $N(v)$, can be expressed as follows:

$\mathbf{h}_{\mathcal{N}(v)}^{k}=\text{ AGG
}_{k}\left(\left\\{\mathbf{h}_{u}^{k-1},\forall{u}\in\mathcal{N}(v)\right\\}\right)\
$ (1)

Here, $\mathbf{h}_{u}^{k-1}$ represents the embedding of node $u$ in the
previous layer. These embeddings of all nodes $u$ in the neighbourhood of $v$
are aggregated into the embedding of node $v$ at layer $k$. This aggregation
process is illustrated in Figure 1 (right) 111In the first layer, i.e. for
$k$=$1$, there is no previous layer $k-1$. In this case, the current node’s
features are used as the previous layer’s node embeddings.. In the GraphSAGE
model, we can use different aggregation methods, including mean, pooling, or
different types of neural networks, e.g. Long Short-Term Memory (LSTM) [5].

The often used mean aggregator function, which we will also use for our
proposed variant of the algorithm, can be expressed as follows:

$\mathbf{h}_{N(v)}^{k}=\sum_{u\in N(v)}\frac{\mathbf{h}_{u}^{k-1}}{\rvert
N(v)\rvert}$ (2)

Here, $\rvert N(v)\rvert$ represents the number of sampled neighbours of node
$v$.

The aggregated embeddings of the sampled neighborhood $\mathbf{h}_{N(v)}^{k}$
are then concatenated with the node’s embedding from the previous layer
$\mathbf{h}_{v}^{k-1}$. After applying model’s trainable parameters
($\mathbf{W}^{k}$, the trainable weight matrix) and passing the result through
a non-linear activation function $\sigma$ (e.g. ReLU), the layer $k$ node $v$
embedding is calculated, as shown in Equation 3.

$\small\mathbf{h}_{v}^{k}=\sigma\left(\mathbf{W}^{k}\cdot\operatorname{CONCAT}\left(\mathbf{h}_{v}^{k-1},\mathbf{h}_{\mathcal{N}(v)}^{k}\right)\right)$
(3)

The final representation (embedding) of node $v$ is expressed as
$\mathbf{z}_{v}$, which is essentially the embedding of the node at the final
layer $K$, as shown in Equation 4. For the purpose of node classification,
$\mathbf{z}_{v}$ can be passed through a sigmoidal neuron or softmax layer.

$\mathbf{z}_{v}=\mathbf{h}_{v}^{K},\ \ \ \ \ \forall v\in\mathcal{V}$ (4)

####  III-B2 Backward Propagation

In the forward propagation phase of the GraphSAGE algorithm, as discussed
above, we assumed the trainable parameters, i.e. the weight matrices
$\mathbf{W}^{k}$ $\forall{k}\in\\{1,...,K\\}$, to be constant. These
parameters are trained and tuned during the backward propagation phase,
similar to other neural network architectures.

Originally, a graph specific loss function was defined and used in the
GraphSAGE algorithm [5]. However, it is also possible to use the standard
neural network loss functions, such as cross entropy, to perform backward
propagation and to update the weights, which can be done by using various
optimisers such as Stochastic Gradient Descent (SGD), and Adam Optimizer
(Adam).

The GraphSAGE algorithm has been successfully applied in a wide range of
applications. However, the algorithm focuses on node features, and is
currently not able to consider edge features. In the context of network
intrusion detection systems (NIDSs), where we are working with flow-based
data, the most relevant features are associated with the flows, i.e. the graph
edges, rather than the nodes. The following section presents E-GraphSAGE, our
proposed variant of GraphSAGE which allows considering the edge (flow)
information in the embedding process. This provides the basis for computing
the corresponding edge embeddings, which then enables edge classification, and
hence the classification of network flows into benign and attack flows.

##  IV E-GraphSAGE

This section introduces Edge-based GraphSAGE (E-GraphSAGE) in two parts. The
first part discusses the E-GraphSAGE model and the modification to the
original GraphSAGE algorithm that were made in order to allow edge embedding
and edge classification to be performed. In the second part, we discuss the
application of E-GraphSAGE for the use case of a network intrusion detection
system (NIDS).

###  IV-A E-GraphSAGE Model

####  IV-A1 Forward Propagation - Edge Embedding

The message passing function in the original GraphSAGE algorithm only
considers node features, and edge features are not taken into consideration
[5]. To include the edge features, it is required to sample and aggregate edge
information of the graph. In addition, the final output of the algorithm needs
to provide an edge embedding rather than the node embedding provided by the
original algorithm. Our proposed E-GraphSAGE algorithm with these
modifications is shown in Algorithm 1.

1

input : Graph $\mathcal{G}(\mathcal{V},\mathcal{E})$;

input edge features
$\left\\{\mathbf{e}_{uv},\forall{uv}\in\mathcal{E}\right\\}$;

input node features $\mathbf{x}_{v}=\\{1,\ldots,1\\}$;

depth $K$;

weight matrices $\mathbf{W}^{k},\forall k\in\\{1,\ldots,K\\}$;

non-linearity $\sigma$;

differentiable aggregator functions ${AGG}_{k}$ ;

2

output : Edge embeddings $\mathbf{z}_{uv},\forall{uv}\in\mathcal{E}$

3 $\mathbf{h}_{v}^{0}\leftarrow\mathbf{x}_{v},\forall v\in\mathcal{V}$

4for  _$k\leftarrow 1\ \textnormal{{to}}\ K$_ do

5 for  _$v\in\mathcal{V}$_ do

6 ${\mathbf{h}_{\mathcal{N}(v)}^{k}\leftarrow\text{
AGG}_{k}\left(\left\\{\mathbf{h}_{uv}^{k-1},\forall{u}\in\mathcal{N}(v),uv\in\mathcal{E}\right\\}\right)}$

7
${\mathbf{h}_{v}^{k}\leftarrow\sigma\left(\mathbf{W}^{k}\cdot\operatorname{CONCAT}\left(\mathbf{h}_{v}^{k-1},\mathbf{h}_{\mathcal{N}(v)}^{k}\right)\right)}$

8 end for

9

10 end for

11$\mathbf{z}_{v}=\mathbf{h}_{v}^{K}$

12for  _$uv\in\mathcal{E}$_ do

13
${\scriptstyle\mathbf{z}_{uv}^{K}\leftarrow{CONCAT}\left(\mathbf{z}_{u}^{K},\mathbf{z}_{v}^{K}\right)}$

14 end for

15

16

Algorithm 1 E-GraphSAGE edge embedding

Key differences to the original GraphSAGE algorithm [5] are in regards to the
algorithm input, the message passing/aggregator function, and the output. The
input includes edge features
$\left\\{\mathbf{e}_{uv},\forall{uv}\in\mathcal{E}\right\\}$, which are, as
mentioned before, missing from the list of inputs in GraphSAGE. Since the
flow-based NIDS datasets only consist of flow (edge) features rather than node
features. Therefore, we are using only provided edge features. We use the
vector $\mathbf{x}_{v}=\\{1,\ldots,1\\}$ to initialise the node features (and
initial node embeddings) and the dimension of all one constant vector is the
same as the number of edge features, shown in Line 1 of the algorithm.

In Line 4, instead of using the standard GraphSAGE node aggregator function
(Equation 1), our newly proposed neighborhood aggregator function creates the
aggregated embeddings of the sampled neighborhood edges at the $k$-th layer,
as shown in Equation 5 below.

$\small\mathbf{h}_{\mathcal{N}(v)}^{k}=\text{ AGG
}_{k}\left(\left\\{\mathbf{h}_{uv}^{k-1},\forall{u}\in\mathcal{N}(v),uv\in\mathcal{E}\right\\}\right)\
$ (5)

Here, $\mathbf{h}_{uv}^{k-1}$ is the representation of edge $uv$ from
$\mathcal{N}(v)$, the sampled neighborhood of node $v$, at layer $k$-$1$. The
set $\\{\forall{u}\in\mathcal{N}(v),uv\in\mathcal{E}\\}$ represents the
sampled edges in the neighborhood $\mathcal{N}(v)$).

In Line 5, the node embedding for node $v$ at layer $k$ is calculated as in
GraphSAGE (Equation 3), but with the critical difference that in the new
algorithm $\mathbf{h}_{\mathcal{N}(v)}^{k}$ is calculated via Equation 5 to
include the edge features.

The final node embeddings at depth $K$, $\mathbf{z}_{v}=\mathbf{h}_{v}^{K}$,
are assigned in Line 8. Finally, the edge embeddings $\mathbf{z}_{uv}^{K}$ for
each edge $uv$ are calculated as the concatenation of the node embeddings of
nodes $u$ and $v$, as shown in Equation 6 below.

$\small\mathbf{z}_{uv}^{K}={CONCAT}\left(\mathbf{z}_{u}^{K},\mathbf{z}_{v}^{K}\right),uv\in\mathcal{E}$
(6)

This represents the final output of the forward propagation stage in
E-GraphSAGE.

####  IV-A2 Backward Propagation

As mentioned, since the use case investigated in this paper is a supervised
learning problem, the backward propagation step is the same as in the original
GraphSAGE algorithm, using the categorical cross-entropy loss function.

###  IV-B E-GraphSAGE NIDS

Figure 2 shows a high level overview of our proposed NIDS based on
E-GraphSAGE. First, the network graph is generated from the network flow data,
and is then fed into the supervised training process of the E-GraphSAGE model
in the next step. In the last step, the edge embeddings are created, which
forms the basis for the classification of edges (network flows) into benign
and attack classes. These three steps are explained in the following
subsections.

Figure 2: Proposed E-GraphSAGE-based NIDS architecture.

####  IV-B1 Network Graph Construction

Network flows (e.g. NetFlow) are a common format for recording network
communication in production networks, and it is also the most common format in
the context of NIDS. The flow records usually consist of fields for
identifying the source and destination of the communication, and the rest of a
record field provides further information on the flow, such as the number of
packets and bytes, flow duration, etc. A graph therefore provides a very
natural way to model such data.

There are various possibilities for defining the end points of a flow and
hence the graph nodes. The method used in this study uses the following 4 flow
fields to identify a graph edge: Source IP Address, Source (L4) Port,
Destination IP Address, and Destination (L4) Port. The first two fields form a
2-tuple that identifies the source node and the two last fields identify the
destination node of the flow. The additional flow fields provide features that
are associated with the corresponding graph edge.

To construct the network graph from the flow data, we mapped the original
source IP addresses to randomly assigned IP addresses in the range from
172.16.0.1 to 172.31.0.1. The reason for this is the fact that in a lot of the
NIDS datasets used in this paper, only a small number of IP addresses were
used as the source of the attacks. The random mapping avoids the potential
problem of the source IP addresses providing an unintentional label for attack
traffic.

Figure 3-a shows a simple example of such a graph with two flow records. Here
the central node $(192.168.1.152:80)$ is exchanging flows with two other
nodes, i.e. $(172.26.185.48:52962)$ and $(172.29.96.53:35866)$.

Figure 3-b shows the visualisation of the graph constructed from the BoT-IoT
dataset, one of six NIDS benchmark datasets considered in this paper, and
discussed in more detail in Section V. Since in our graph construction all
remaining flow record fields were assigned to the edge, the graph nodes are
featureless. It has been shown [13] that in classification scenarios with
featureless nodes, it can be beneficial to assign a one vector, a vector with
all values of one, to all nodes. As shown in Algorithm 1, we are using this
method in our algorithm, with the dimensionality of the all one constant
vector in each node depending on the number of edge features.

(a)

(b)

Figure 3: Network Graph Construction and Visualisation: a) A sample network
graph for two flows, and b) sample graph visualisation for the BoT-IoT
dataset.

####  IV-B2 E-GraphSAGE Training

The neural network model we use in our implementation consists of two
E-GraphSAGE layers ($K=2$), which means that neighbour information is
aggregated from a two-hop neighbourhood. For the aggregation function $AGG$,
as shown in Equation 5, we use the mean function which simply finds the
element-wise mean of the edge embeddings in the sampled neighborhood. The
definition of the mean aggregator function in E-GraphSAGE is provided in
Equation 7 below.

$\mathbf{h}_{\mathcal{N}(v)}^{k}=\sum_{\begin{subarray}{c}{u}\in\mathcal{N}(v),\\\
uv\in\mathcal{E}\end{subarray}}\frac{\mathbf{h}_{uv}^{k-1}}{\rvert
N(v)\rvert_{e}}$ (7)

Here, $\rvert N(v)\rvert_{e}$ represents the number of edges in the sampled
neighborhood, and $\mathbf{h}_{uv}^{k-1}$ represents their embedding at layer
$k-1$. For our implementation, we chose full neighborhood sampling, which
means information from the full set of edges in a node’s neighbourhood is
aggregated.

In both E-GraphSAGE layers, for the hidden feature sizes of each layer
represented in Equation 3, we use a size of 128 hidden units, which is also
the dimension of the node embeddings. For the nonlinear transformation
($\sigma$ in Equation 3), we use the ReLU activation function, and for the
purpose of regularisation, we use a dropout mechanism with a rate of $0.2$
between the two E-GraphSAGE layers. We chose the cross-entropy loss function,
and gradient descent in the backward propagation phase is performed using the
Adam optimizer.

When node embeddings are generated in the last E-GraphSAGE layer, they are
converted to the corresponding edge embeddings (Line 10 of Algorithm 1). Since
the edge embeddings are created via concatenation of two node embeddings, the
size of the edge embeddings is 256 dimensions.

The output edge embeddings pass through a Log Softmax layer, which makes it
possible to compare the output of the algorithm to the labels provided by the
NIDS datasets and to tune the trainable model parameters in the backward
propagation phase.

####  IV-B3 Edge Classification

After the model parameters have been tuned in the training process, the model
is ready for evaluation via the classification of unseen test samples. The
test flow records are also converted to graphs, passed through the trained
E-GraphSAGE layers, from which the edge embeddings are calculated. They are
then converted to class probabilities in the final Log Softmax layer, and are
finally compared with the true class labels in order to compute the
classification evaluation performance metrics.

##  V Datasets

For the evaluation of the GNN-based NIDS proposed in this paper, we use six
publicly available NIDS datasets, which consist of different types of labelled
attack flows as well as benign network flows. The first three datasets,
consisting of UNSW-NB15 [14], ToN-IoT [15] and BoT-IoT [16], have proprietary
formats and feature sets, and have been widely used to evaluate Machine
Learning based network intrusion detection systems. In addition, we also
consider the three recently created variants of these datasets, where the
respective format is translated to NetFlow with a common feature set, i.e. NF-
UNSW-NB15[8], NF-TON-IoT [8] and NF-BoT-IoT [8]. A brief overview of these
datasets is provided in the following.

  * •

UNSW-NB15: N. Moustafa et al. [14] created the UNSW-NB15 dataset to address
the shortcomings of the existing benchmark datasets. The authors used IXIA
tools to generate a comprehensive set of attack traffic in a complex network
environment, consisting of nine main attack families. The Argus Bro-IDS
network-monitoring tool was used to generate 49 features traffic class labels.
The dataset consists of 2,218,761 (87.35%) benign flows and 321,283 (12.65 %)
attack flows, with a total size of 2,540,044 flows.

  * •

ToN-IoT: This is a relatively new and extensive dataset that was generated in
2019 by Abdullah et al.[15], which includes different types of IoT data, such
as operation system logs, telemetry data of IoT/IIoT services, as well as IoT
network traffic collected from a medium-scale network at the Cyber Range and
IoT Labs at the UNSW Canberra (Australia). In this paper, only the network
traffic component of the dataset is used. The Bro-IDS network monitoring tool
was used to generate the dataset’s 44 network flow features. The dataset
consists of 796,380 (3.56%) benign flows and 21,542,641 (96.44%) attack flows,
with a total of 22,339,021 flows.

  * •

BoT-IoT: This is another recent dataset with a specific on IoT networks,
created by Koroniotis et al. [16] in 2019. The authors simulated IoT services
such as water stations, by using the Node-red tool [17] and generated the
corresponding IoT traffic. The Argus tool was consequently used for feature
extraction. This dataset is comprised of 6 types of attacks and a total of 47
features with corresponding class labels. The dataset contains only 477
(0.01%) benign flows and 3,668,045 (99.99%) attack flows, with a total of
3,668,522 flows.

  * •

NF-UNSW-NB15, NF-ToT-IoT and NF-BoT-IoT: A lack of a standard format and
feature set among the various NIDS datasets has made it very difficult to
compare the performance of ML-based network traffic classifiers across
different datasets, and to evaluate their ability to generalise to different
network scenarios. Sarhan et al. [8] have addressed this problem by providing
the NetFlow version of the above mentioned three NIDS datasets. The authors of
[8] used the raw packet capture (pcap) files of the original NIDS datasets and
converted them to the NetFlow format via the nprobe [18] tool and selecting 12
features to be extracted, resulting in the new variants of the original
datasets, i.e. NF-UNSW-NB15, NF-ToT-IoT and NF-BoT-IoT. The NF-UNSW-NB15
dataset has a total number of 1,623,118 flows, of which 72,406 (4.46 %) are
attack flows and 1,550,712 (95.54 %) are benign flows. In NF-ToT-IoT, the
total number of network flows is 1,379,274, out of which 1,108,995 (80.4 %)
are attack flows and 270,279 (19.6 %) are benign flows. Finally, the NF-BoT-
IoT dataset has a total number of 600,100 flows, out of which 586,241 (97.69
%) are attack flows and 13,859 (2.31 %) are benign flows.

TABLE I: Evaluation metrics utilised in this study. Metric | Definition
---|---
Recall | $\frac{TP}{TP+FN}$
Precision | $\frac{TP}{TP+FP}$
F1-Score | $2\times\frac{Recall\times Precision}{Recall+Precision}$
Accuracy | $\frac{TP+TN}{TP+FP+TN+FN}$
False Alarm Rate (FAR) | $\frac{FP}{FP+TN}$

##  VI Experimental Results

To evaluate the performance of the proposed neural network model, the standard
metrics listed in Table I are used, where $TP$, $TN$, $FP$ and $FN$ represent
the number of True Positives, True Negatives, False Positives and False
Negatives respectively. First, the results of the binary classification
problem, where the aim is to distinguish between attack and benign traffic,
are presented. Subsequently, we present the results of the multiclass
classification experiments, where the aim was to identify the specific attack
type of each flow.

###  VI-A Binary Classification Results

The datasets used in this paper all include two sets of labels. The first
label determines if a flow record belongs to a benign or attack class, and the
second label identifies the attack type. We use the first label set for binary
classification, and the second set for multiclass classification.

We use the full datasets, with the exception of the ToN-IoT dataset, where we
considered a randomly sampled subset with 10% of the original size, due to the
very large size of the original dataset. In regards to training and evaluation
data split, 70% of the flow records of each dataset were selected for training
and 30% were reserved for testing and evaluation.

TABLE II: E-GraphSAGE binary classification results.

Dataset | Accuracy | Precision | F1-Score | Recall (DR) | FAR
---|---|---|---|---|---
UNSW-NB15 | 99.35% | 0.97 | 0.97 | 97.51% | 0.38%
NF-UNSW-NB15 | 98.75% | 0.87 | 0.86 | 84.31% | 0.58%
BoT-IoT | 99.99% | 1.00 | 1.00 | 99.99% | 0.00 %
NF-BoT-IoT | 93.57% | 1.00 | 0.97 | 93.43% | 0.38 %
ToN-IoT | 97.87% | 1.00 | 0.99 | 97.86% | 1.92%
NF-ToN-IoT | 99.69% | 1.00 | 1.00 | 99.85% | 0.15%

As mentioned above, the experimental evaluation was based on three original
datasets with proprietary flow formats and feature sets, as well as their
corresponding NetFlow versions. As shown in [8], even though the original
dataset and its NetFlow counterpart represent essentially the same network
events, the performance of a classifier for the two datasets versions can vary
significantly, due to the different feature sets.

Table II presents the detailed results of our E-GraphSAGE classifier for the
binary classification case, showing Accuracy, Precision , F1-Score, Recall and
FAR, for the considered six benchmark datasets. As can be seen in the table,
the E-GraphSAGE classifier performs very well across all the different
performance metrics. Since the considered datasets are generally highly
imbalanced, the F1-Score is a more relevant performance metric. We use the
F1-Score, in addition to Accuracy, to compare our classifier with the state-
of-the-art, i.e. the best classification results reported in the literature
for each of the six NIDS datasets.

Table III shows the corresponding results of our E-GraphSAGE classifier
compared to the top one or two related works for each dataset in terms of
Accuracy and F1-Score. As can be observed from the table, in regards to
Accuracy, E-GraphSAGE outperforms the best reported classifiers for four out
of the six considered datasets. For the two datasets where this is not the
case, i.e. BoT-IoT and NF-BoT-IoT, the difference to the top performing
classifier is only 0.01% and 0.25% respectively.

TABLE III: Binary classification performance comparison with the state-of-art algorithms. Method | Dataset | Accuracy | F1
---|---|---|---
Proposed E-GraphSAGE | UNSW-NB15 | 99.35% | 0.97
Two-Level Hybrid Model [1] | UNSW-NB15 | 99.00% | 0.99
CNN-WDLSTM [3] | UNSW-NB15 | 97.17% | 0.97
Proposed E-GraphSAGE | NF-UNSW-NB15 | 98.75% | 0.86
Extra Tree Classifier [8] | NF-UNSW-NB15 | 98.62% | 0.85
Proposed E-GraphSAGE | BoT-IoT | 99.99% | 1.00
Extra Tree Classifier [8] | BoT-IoT | 100.00% | 1.00
XGBoost [7] | BoT-IoT | 99.99% | 0.99
Proposed E-GraphSAGE | NF-BoT-IoT | 93.57% | 0.97
Extra Tree Classifier [8] | NF-BoT-IoT | 93.82% | 0.97
Proposed E-GraphSAGE | ToN-IoT | 97.87% | 1.00
Ensemble [19] | ToN-IoT | 96.35% | 0.95
Proposed E-GraphSAGE | NF-ToN-IoT | 99.69% | 1.00
Extra Tree Classifier [8] | NF-ToN-IoT | 99.66% | 1.00
Figure 4: Comparing E-GraphSAGE with the state-of-art algorithms in binary
classification.

The performance comparison of E-GraphSAGE with the best reported results in
the literature in regards to F1-Score is summarised and visualised in Figure
4. E-GraphSAGE outperforms or equals the state-of-the-art, i.e. the best
reported results for each respective dataset, with the only exception of the
UNSW-NB15 dataset, where the difference to the highest achieved F1-Score is
only 2%.

These results demonstrate the power of our GNN-based approach for traffic
classification and network intrusion detection. In contrast to most related
works, where a new ML-based NIDS classifier is evaluated on one, two or a
maximum three datasets, we have demonstrated the performance of our
E-GraphSAGE approach across six significantly different NIDS benchmark
datasets. The consistently solid results demonstrate the robustness of our
GNN-based approach and its ability to generalise across different types of
network traffic patterns, feature sets and attack types. We believe this is
due to the ability of the GNN architecture to capture the information in the
inherent graph structure of the flow based NIDS datasets.

(a)

(b)

Figure 5: Visualisation of dimensionality reduction a) Sample of BoT-IoT raw
validation data, b)Sample of edge embeddings generated by E-GraphSAGE
(Binary).

To get an intuitive understanding of the excellent and robust performance of
the E-GraphSAGE classifier, we investigated and visualised the edge embeddings
generated by the algorithm, based on the BoT-IoT dataset.

For this, we generate two visualisations. First, we take the sample ‘raw’ BoT-
IoT dataset and convert it into a graph format, with the relevant flow
features mapped to graph edge features. We then applied the Uniform Manifold
Approximation and Projection (UMAP) dimensionality reduction algorithm to map
the high dimensional data down to two dimensions, for the purpose of
visualisation. The result is shown in Figure 5 a, where the blue dots indicate
attack flows, and the orange dots represent normal or benign flows. We clearly
see the high imbalance of the dataset, with most flows being attack flows and
only a small number of benign flows, i.e. orange dots. It is clear from the
figure, and as expected, that there is no clear separation between the normal
and attack flows in the two-dimensional space.

In a second experiment, we take the sample BoT-IoT dataset in graph format,
and feed it into the pre-trained E-GraphSAGE model (binary classifier), which
then computes the corresponding edge embeddings. We then take these embeddings
and also feed them onto the UMAP algorithm to map them into a two dimensional
space. The result is shown in Figure 5 b. In contrast to Figure 5 a, we now
see a clear separation of attack flows and normal/benign flows. This
illustrates the ability of edge embedding mechanism of the E-GraphSAGE
algorithm to leverage the inherent graph structure of the flow based NIDS
data, and to be able to separate attack traffic from normal traffic in the
embedding space, which in turn results in a high classification performance.

###  VI-B Multiclass Classification Results

TABLE IV: Results of multiclass classification by E-GraphSAGE on UNSW-NB15 dataset families. | UNSW-NB15 | NF-UNSW-NB15
---|---|---
Class Name | DR | F1-Score | DR | F1-Score
Benign | 99.70% | 1.00 | 97.71% | 0.99
Analysis | 4.48% | 0.08 | 33.39% | 0.18
Backdoor | 7.30% | 0.13 | 84.02% | 0.23
DoS | 41.30% | 0.40 | 19.41% | 0.18
Exploits | 78.42% | 0.72 | 51.99% | 0.62
Fuzzers | 58.98% | 0.62 | 79.77% | 0.53
Generic | 98.58% | 0.99 | 44.82% | 0.43
Reconnaissance | 78.67% | 0.84 | 82.75% | 0.80
Shellcode | 86.98% | 0.78 | 99.02% | 0.31
Worms | 13.46% | 0.20 | 91.30% | 0.08
Weighted Average | 98.15% | 0.98 | 96.17% | 0.97
TABLE V: Results of multiclass classification by E-GraphSAGE on BoT-IoT dataset families | BoT-IoT | NF-BoT-IoT
---|---|---
Class Name | DR | F1-Score | DR | F1-Score
Benign | 100.00% | 0.98 | 99.45% | 0.42
DDoS | 99.99% | 1.00 | 40.82% | 0.39
DoS | 99.99% | 1.00 | 57.13% | 0.47
Reconnaissance | 99.98% | 1.00 | 84.50% | 0.92
Theft | 93.75% | 0.97 | 99.83% | 0.39
Weighted Average | 99.99% | 1.00 | 78.16% | 0.81

We are now considering the multiclass classification problem, where the
classifier aims to distinguish between different types of attacks as well as
benign traffic. This is a much harder problem than in the binary case. For the
evaluation of the E-GraphSAGE classifier in the multiclass scenario, we
considered the same six NIDS datasets as for the binary case. Depending on the
NIDS dataset, the number of attack classes range between 5 and 10. Table IV
shows the Detection Rate (DR) and the F-Score for each individual attack
class, including the benign class, for the UNSW-NB15 and its Netflow
counterpart (NF-UNSW-NB15). We observe that the results vary significantly
across the different flow types, with some attack types such as ‘Generic’ and
’Shellcode’ achieving a high Detection Rate of 98.58% and 86.98% respectively,
in contrast to ‘Backdoor’ and ‘Analysis’ with corresponding values as low as
7.30% and 4.48%. This variation among different attack types corresponds with
results reported in the literature [1][3].

The results also show some differences between the two variants of the UNSW-
NB15 datasets, which is not unexpected, given the very different feature sets.
The last row of the table also shows the weighted averages of the performance
metrics, for both dataset variants. Overall, a weighted F1-Score of 0.98 for
UNSW-NB15 and 0.97 for UNSW-NB15 represent solid results for the E-GraphSAGE
classifier.

Tables V and VI show the corresponding results for the BoT-IoT and ToN-IoT
datasets, and their respective Netflow counterparts. For the BoT-IoT dataset
with its original format and feature set, a very weighted high Detection Rate
and F1-Score is achieved for all the 5 traffic classes (4 attack classes plus
benign class), with a weighted average of 99.99% and 1.00 respectively. The
corresponding numbers for the NF-BoT-IoT dataset are significantly lower, with
a weighted average DR value of 78.16% and weighted F1-Score of 0.81. The
results for the ToN-IoT datasets show a high weighted average Detection Rate
(86.78%) and F1-Score (0.87), but the values vary significantly for the
individual traffic classes, similar to the UNSW-NB15 dataset. As for the BoT-
IoT dataset, we observe a significantly lower classification performance for
the NetFlow variant of the BoT-IoT dataset. This can be explained by the very
different nature of the feature set. While the original feature sets include
specifically engineered features to detect the attack classes included in the
dataset, the NetFlow variant consist of relatively simple and generic
features.

TABLE VI: Results of multiclass classification by E-GraphSAGE on ToN-IoT datasets families. | ToN-IoT | NF-ToN-IoT
---|---|---
Class Name | DR | F1-Score | DR | F1-Score
Benign | 88.12% | 0.91 | 98.86% | 0.92
Backdoor | 5.06% | 0.08 | 98.38% | 0.99
DDoS | 96.94% | 0.98 | 52.35% | 0.68
DoS | 96.08% | 0.73 | 0.00% | 0.00
Injection | 88.94% | 0.83 | 93.15% | 0.71
MIMT | 87.43% | 0.18 | 22.88% | 0.28
Ransomware | 98.55% | 0.94 | 96.49% | 0.23
Password | 89.15% | 0.91 | 19.92% | 0.25
Scanning | 75.84% | 0.85 | 15.32% | 0.13
XSS | 92.08% | 0.95 | 0.00% | 0.00
Weighted Average | 86.78% | 0.87 | 67.16% | 0.63

As in the binary classification case, we provide a comparison of our
E-GraphSAGE-based NIDS with the state-of-the-art. Table VII shows the average
multiclass Accuracy and weighted F1-Scores of E-GraphSAGE compared with the
top one or two best reported results in the literature, depending on
availability. We observe that E-GraphSAGE outperforms or equals the best
related works for four out of six considered datasets in terms of Accuracy,
and in the other two cases, the difference to the top performer is only
minimal, i.e. 0.28% in the case of UNSW-NB15 and 1.45% for NF-UNSW-NB15. The
comparison in regards to F1-Score is further summarised and visualised in
Figure 6. We see that E-GraphSAGE outperforms or equals the top performing
state-of-the-art classifiers for five out of the six NIDS datasets, and in the
case of NF-UNSW-NB15, the difference to the top performing Extra Tree
classifier is only 0.01, i.e. with 0.97 for E-GraphSAGE and 0.98 as reported
in [8].

TABLE VII: Performance of multiclass classification by E-GraphSAGE compared with the state-of-art algorithms. Method | Dataset | Accuracy | Weighted F1
---|---|---|---
Proposed E-GraphSAGE | UNSW-NB15 | 98.15% | 0.98
CNN-WDLSTM [3] | UNSW-NB15 | 98.43% | 0.98
Two-Level Hybrid Model [1] | UNSW-NB15 | 97.00% | 0.97
Proposed E-GraphSAGE | NF-UNSW-NB15 | 96.17% | 0.97
Extra Tree Classifier [8] | NF-UNSW-NB15 | 97.62% | 0.98
Proposed E-GraphSAGE | BoT-IoT | 99.99% | 1.00
XGBoost [7] | BoT-IoT | 99.96% | 0.97
Decision Tree [7] | BoT-IoT | 99.96% | 0.96
Proposed E-GraphSAGE | NF-BoT-IoT | 78.16% | 0.81
Extra Tree Classifier [8] | NF-BoT-IoT | 73.58% | 0.77
Proposed E-GraphSAGE | ToN-IoT | 86.78% | 0.87
Extra Tree Classifier [8] | ToN-IoT | 84.61% | 0.87
Proposed E-GraphSAGE | NF-ToN-IoT | 67.16% | 0.63
Extra Tree Classifier [8] | NF-ToN-IoT | 56.34% | 0.60

Overall, E-GraphSAGE performs consistently very well compared to the state-of-
the-art, in particular if we consider our stringent comparison methodology,
where we pick the best performing classifier for each of the six individual
NIDS datasets, and perform a pairwise comparison with E-GraphSAGE. This is in
contrast to the typical approach where a classifier is compared with related
approaches on one, two or a maximum of three common datasets.

As for the binary classification case, we aim to get an intuitive
understanding of the ability of Graph Neural Networks, and E-GraphSAGE in
particular, to use the inherent graph structure of flow-based network datasets
to separate different traffic and attack classes. We used the same approach as
in Section VI-A, taking the raw BoT-IoT graph dataset, and running it through
UMAP dimensionality reduction algorithm to obtain a 2D representation of the
data. This is shown in Figure 7a. Each dot represents a flow, which is
coloured according to its traffic class. We observe there is no clear
separation of traffic classes and the different data points seem randomly
distributed. For comparison, we take the BoT-IoT dataset in graph
representation, and feed into a E-GraphSAGE model which has been pre-trained
as multiclass classifier. As a result we get the corresponding (high
dimensional) edge embedding of the dataset. In order to be able to visualise
the data, we again apply the UMAP algorithm to map the data into a 2D space.
The result is shown in Figure 5b, which clearly illustrates the ability of
E-GraphSAGE to separate the different traffic and attack types into clearly
discernible clusters, leading to highly accurate classification.

###  VI-C Classification Runtime Results

To demonstrated the computational efficiency of the E-GraphSAGE model, we
measured the average classification time per flow in µs. The results for the
six considered datasets are shown in Table VIII. The measurements were
performed on a single NVIDIA RTX 6000 GPU. As the results show, the
E-GraphSAGE model allows for very efficient classification of flows, and hence
the detection of malicious network traffic, in both the binary and multiclass
classification case.

TABLE VIII: E-GraphSAGE classification runtime.

Dataset | Binary Classification (µs) | Multiclass Classification (µs)
---|---|---
UNSW-NB15 | 0.16 | 0.16
NF-UNSW-NB15 | 0.13 | 0.09
BoT-IoT | 0.24 | 0.51
NF-BoT-IoT | 0.16 | 0.14
ToN-IoT | 0.14 | 0.14
NF-ToN-IoT | 0.14 | 0.21

Figure 6: Comparing multiclass classification performance of E-GraphSAGE with
the state-of-art algorithms.

(a)

(b)

Figure 7: Visualisation of dimensionality reduction a) Sample of BoT-IoT raw
validation data, b)Sample of edge embeddings generated by E-GraphSAGE
(Multiclass).

##  VII Conclusions

This paper presents a novel approach to network intrusion detection system
based on Graph Neural Networks. For this, we proposed E-GraphSAGE, a variant
of the well-known GraphSAGE algorithm, which enables edge embedding and edge
classification. We demonstrate how E-GraphSAGE can be used to build an NIDS
which allows the detection of malicious network flows. To the best of our
knowledge, this represents the first implementation and extensive evaluation
of an NIDS based on Graph Neural Networks. Our extensive experimental
evaluation based on six NIDS benchmark datasets shows that the E-GraphSAGE
based NIDS performs exceptionally well, and outperforms or matches the
performance of the best reported related systems, with only a couple of minor
exceptions. Our results demonstrate the potential of GNN-based approaches, and
provide motivation for promising future research in this space. For example,
we believe that with a more thorough and systematic exploration of the
hyperparameter space, the performance of GNN-based network intrusion detection
can be further improved.

## References

  * [1] Imtiaz Ullah and Qusay Mahmoud  “A Two-Level Hybrid Model for Anomalous Activity Detection in IoT Networks”  In _2019 16th IEEE Annual Consumer Communications Networking Conference (CCNC)_ , 2019, pp. 1–6  DOI: 10.1109/CCNC.2019.8651782
  * [2] Tamer Aldwairi, Dilina Perera and Mark A Novotny  “An evaluation of the performance of Restricted Boltzmann Machines as a model for anomaly network intrusion detection”  In _Computer Networks_ 144, 2018, pp. 111–119  URL: http://www.sciencedirect.com/science/article/pii/S1389128618306005
  * [3] Mohammad Hassan et al.  “A Hybrid Deep Learning Model for Efficient Intrusion Detection in Big Data Environment”  In _Information Sciences_ 513, 2019  DOI: 10.1016/j.ins.2019.10.069
  * [4] Z. Wu et al.  “A Comprehensive Survey on Graph Neural Networks”  In _IEEE Transactions on Neural Networks and Learning Systems_ 32.1, 2021, pp. 4–24  DOI: 10.1109/TNNLS.2020.2978386
  * [5] William L. Hamilton, Rex Ying and Jure Leskovec  “Inductive representation learning on large graphs”  In _Advances in Neural Information Processing Systems_ , 2017  arXiv:1706.02216
  * [6] H. He et al.  “A Novel Multimodal-Sequential Approach Based on Multi-View Features for Network Intrusion Detection”  In _IEEE Access_ 7, 2019, pp. 183207–183221  DOI: 10.1109/ACCESS.2019.2959131
  * [7] Muhammad Aminu Lawal, Riaz Ahmed Shaikh and Syed Raheel Hassan  “An Anomaly Mitigation Framework for IoT Using Fog Computing”  In _Electronics_ 9.10, 2020  DOI: 10.3390/electronics9101565
  * [8] Mohanad Sarhan, Siamak Layeghy, Nour Moustafa and Marius Portmann  “NetFlow Datasets for Machine Learning-based Network Intrusion Detection Systems”, 2020
  * [9] Qingsai Xiao et al.  “Towards Network Anomaly Detection Using Graph Embedding”  In _Computational Science – ICCS 2020_ Cham: Springer International Publishing, 2020, pp. 156–169
  * [10] Jiawei Zhou, Zhiying Xu, Alexander Rush and Minlan Yu  “Automating Botnet Detection with Graph Neural Networks”  In _4th Workshop on Machine Learning and Systems (MLSys)_ , 2020
  * [11] M.. Bronstein et al.  “Geometric Deep Learning: Going beyond Euclidean data”  In _IEEE Signal Processing Magazine_ 34.4, 2017, pp. 18–42  DOI: 10.1109/MSP.2017.2693418
  * [12] Jie Zhou et al.  “Graph Neural Networks: A Review of Methods and Applications”  In _ArXiv_ abs/1812.08434, 2018
  * [13] Yepeng Yao et al.  “Marrying Graph Kernel with Deep Neural Network: A Case Study for Network Anomaly Detection”  In _Computational Science – ICCS 2019_ Cham: Springer International Publishing, 2019, pp. 102–115
  * [14] Nour Moustafa and Jill Slay  “UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set)”  In _2015 Military Communications and Information Systems Conference, MilCIS 2015 - Proceedings_ , 2015  DOI: 10.1109/MilCIS.2015.7348942
  * [15] A. Alsaedi, N. Moustafa, A. Z. and A. Anwar  “TON_IoT Telemetry Dataset: A New Generation Dataset of IoT and IIoT for Data-Driven Intrusion Detection Systems”  In _IEEE Access_ 8, 2020, pp. 165130–165150  DOI: 10.1109/ACCESS.2020.3022862
  * [16] Nickolaos Koroniotis, Nour Moustafa, Elena Sitnikova and Benjamin Turnbull  “Towards the development of realistic botnet dataset in the Internet of Things for network forensic analytics: Bot-IoT dataset”  In _Future Generation Computer Systems_ , 2019  DOI: 10.1016/j.future.2019.05.041
  * [17] “Low-code programming for event-driven applications”  In _Low-code programming for event-driven applications_ , 2021  URL: https://nodered.org/
  * [18] “An Extensible NetFlow v5/v9/IPFIX Probe for IPv4/v6”  In _An Extensible NetFlow v5/v9/IPFIX Probe for IPv4/v6_ , 2021  URL: https://www.ntop.org/products/netflow/nprobe/
  * [19] “An ensemble learning and fog-cloud architecture-driven cyber-attack detection framework for IoMT networks”  In _Computer Communications_ 166, 2021, pp. 110–124